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Abstract 

Random coding, along with various standard techniques such as coded time-sharing, rate-splitting, superposition coding, 
and binning, are traditionally used in obtaining achievable rate regions for multi-terminal networks. The error analysis of such 
an achievable scheme relies heavily on the properties of strongly joint typical sequences and on bounds of the cardinality of 
^S) ', typical sets. In this work, we obtain an achievable rate region for a general (i.e. an arbitrary set of messages shared amongst 

encoding nodes, which transmit to arbitrary decoding nodes) memoryless, single-hop, multi-terminal network without feedback or 
• cooperation by introducing a general framework and notation, and carefully generalizing the derivation of the error analysis. We 

■ show that this general inner bound may be obtained from a chain graph representation of the encoding operations. This graph 

representation captures the statistical relationship among codewords and allows one to readily obtain the rate bounds that define the 
achievable rate region. The proposed graph representation naturally leads to the derivation of all the achievable schemes that can 
be generated by combining classic random coding techniques for any memoryless network used without feedback or cooperation. 
We also re-derive a few achievable regions for classic multi-terminal networks, such as the multi-access channel, the broadcast 
\f~^ \ channel, and the interference channel, to show how this new representation allows one to quickly consider the possible choices 

of encoding/decoding strategies for any given network and the distribution of messages among the encoders and decoders. 



I. Introduction 

' Random coding was first introduced by Shannon in his seminal 1948 paper IT] to prove the channel coding theorem: the 
essence of this proving method is to analyze the performance of an ensemble of randomly generated codes, rather than of just 
one good code. By deriving the average performance of an ensemble of codes, we are assured that a good code exists that 
performs above the average of the ensemble. 
In Shannon's words: 

0^ ■ The method of proving [the capacity of a discrete channel with noise] is not by exhibiting a coding method having 

\ the desired properties, but by showing that such a code must exist in a certain group of codes. 

In random coding, codewords are generated by drawing symbols in an independent, identically distributed (i.i.d.) fashion 
. from a prescribed distribution; the performance of the ensemble of codes is then analyzed as a function of the block-length, 
I ' which is eventually taken to infinity. Thanks to the i.i.d. symbols, and a block-length which tends to infinity, it is possible 
. to derive the asymptotic performance of the ensemble of codes using the properties of jointly typical sets 111. This proving 
1^ ' technique was originally developed for the point-to-point channel but is easily extended to multi-user channels by introducing 
multiple codebooks, one for each message to be transmitted. Multiple codebooks can further be organized in complex schemes 
that takes into account the structure of the network and the distribution of the messages among the users. Coded time-sharing, 
rate-spitting, superposition coding, binning, Markov encoding, compress and forward, decode, and forward, are some of the 
strategies that have been developed for multi-terminal channels using the random coding proof technique. Given that all 
achievability schemes tend to use a combination of "standard" techniques applied in different fashions (leading to different 
dependencies amongst codewords), one might expect to be able to derive a general achievability scheme for a large class of 
networks. 

The key bounding techniques to analyze the error probability of transmission schemes are presented in an unified fashion 
by Csiszar and Korner [2 Ch. 1.2] and, more recently, by Kramer [3, Ch. 1] and El Gamal \^ Ch. 1]. Two main results are 
the "covering lemma" and the "packing lemma". The "covering lemma" describes how to cover a typical set with subsets so 
that all the typical sequences are contained in at least one subset and it is used to bound the number of codewords available 
at the encoder The "packing lemma", on the other hand, deals with the number subsets that can be packed in a typical set so 
that no typical sequence is contained in more than one subset and it is used to bound the number of typical sequences that 
can be distinguished at the receiver By applying these two lemmas, it is possible to derive the achievable region of a number 
of transmission schemes based on random coding arguments. 

In the following we consider two such random coding arguments: superposition coding and binning superposition coding 
and binning are encoding techniques that introduce dependency among codewords utilized by different users. Superposition 
coding is a method of stacking a codebook on top of another codebook and it was first introduced by Cover |6| to derive 
an achievable region for the broadcast channel. In superposition coding a first message is conveyed using a "base" codebook 
of i.i.d. codewords as in the point to point channel while a second message is transmitted using a "top" codebook that is 



'sometimes refen'ed to as Cover random binning |4| or Gel'fand-Pinsker coding (5) 
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generated for each codeword in the base codebook. The codeword of the top codebook is obtained as i.i.d. draws from a 
conditional distribution that depends on the chosen base codeword. This way of generating top and bottom codebooks presents 
the advantage that, once the bottom codeword has been decoded, the top codeword can be decoded by using the knowledge of 
the base codeword and of the conditional probabiUty of the top codeword. This facilitates the decoding of the top codeword 
since it reduces the probability of the decoder mistakenly choosing an incorrect top codeword. Bergamas and Gallager Q, 
lH) proved that superposition coding achieves capacity for the degraded broadcast channel, that is, a broadcast channel where 
one of the channel outputs is a degraded version of the other. The capacity of this channel is achieved by superposing the 
message for the non-degraded user on top of the message for the degraded user. Moreover the non-degraded user decodes both 
messages while the degraded user decodes only the base message. Since the non-degraded user also decodes the degraded 
message, it is able to boost the decoding of its own message by using the knowledge of the degraded message and its impact 
on the conditional distribution of the top message, thus achieving capacity. 

Binning is similar to superposition coding in that it aims at introducing dependency among codewords of different users; this 
dependency is not introduced at the codebook level, but rather during the encoding procedure. Binning was devised by Sleepian 
and Wolf ||9) to solve the problem of distributed source coding and it was later used by Marton to define an achievable region 
for the general broadcast channel [10|. As in superposition coding, two messages are encoded using a "top" and a "base" 
codebook. In the base codebook, each codeword is associated to a message to be sent while in the second codebook multiple 
codewords are used to transmit the same massage. Codewords associated to the same message are placed in a bin and the 
encoder chooses among the among the codewords in the bin for a codewords that appears as if conditionally dependent from 
the base codeword. If such codeword can be found it is transmitted over the channel. The advantage provided by binning is 
that it introduced dependency among the codewords of different users without requiring that a user decode both codewords 
as in binning. The non-intended codeword represents interference at a given decoder and binning reduces the effect of such 
interference by correlating the interference with the transmitted codeword. 

Superposition coding and binning are two random coding techniques to generate transmission codebooks and encoding 
messages. The rate regions that can be achieved with these random coding techniques can be further enlarged by considering 
two conceptually simple transmission strategies: coded time-sharing and rate-splitting. Coded time-sharing was originally 
introduced by Han and Kobayashi 1 1 1 1 when deriving an achievable region for the interference channel and it consists of 
utilizing different codebooks at different time instances according to a random schedule. Coded time-sharing generalizes time- 
division and frequency division multiple access and it achieves the convex hull of the time shared achievable regions and 
possible larger regions (2^, pp. 288-290]. Although time-sharing is sometimes not needed to achieve capacity, it is necessary 
to show the convexity of the achievable region based on superposition coding for the broadcast channel fT2|. ifTsl and for the 
interference channel in strong interference |[T4l . ifTSl . 

As for coded time-sharing, rate-splitting was also introduced by Han and Kobayashi in 1 1 1 1 and it consists of splitting a 
message of one user into multiple sub-messages to be decoded by a larger set of decoders than the original intended decoder. 
Since each of the sub-messages is still decoded by the original decoder, the transmission rate is unchanged but by splitting the 
message into subparts it is possible to utilize a larger set of transmission strategies. In |11 1 rate-splitting is used to derive a new 
achievable region for the classic interference channel. The message at each transmitter is divided into private and public parts 
and the private messages are superposed over the public ones. Each transmitter successively decodes both public messages 
and the private message from the intended transmitter. Without rate-splitting, the only available transmission strategy at the 
encoders is to transmit the two messages in two independent codewords and each decoder can then either decode one codeword 
or two. By splitting the messages into public and private parts it is possible to construct a more compUcated transmission 
strategy without altering the rate at which each message is being transmitted. 

Although many other encoding strategies have been proposed in information theory, these simple four strategies achieve 
capacity for a large number of one-hop channels with no feedback or user cooperation. 

Tab. I] summarizes several classic capacity results for some of such channels and indicates which of the four transmission 
strategies, superposition coding, binning, rate-splitting and time-sharing is necessary to achieve capacity. 

In this paper we introduce an achievable scheme involving superposition coding, binning, rate-splitting, and coded time- 
sharing valid for a general one-hop channel without feedback. This achievable scheme is defined by the random variables 
representing different codewords and by the factorization of the joint distribution among these random variables. We develop a 
graph representation of the factorization of the joint distribution based on Markov graphs ll27l . Il28l : in particular we define a 
Markov chain graph with two types of edges, one representing superposition coding and one representing binning. In information 
theory, a similar attempt to capture the relationship between random variables is the functional dependence graph |3, App. 8]. 
Despite fundamental similarities, the functional dependence graph is used mainly to represent the Markov relationship between 
random variables, while, in our case, the Markov chain graph describes a transmission scheme and the relationship between 
codewords of different users. 

By building upon the fundamental results in random coding theory and graph theory, we define a formal representation and 
a standard notation for a general achievable scheme as well as the derivation of the achievable region. Our ultimate goal is 
to define a form of "automatic rate region generator" which outputs the best known random coding achievable rate region for 
any channel of choice. 
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TABLE I 

Capacity results involving superposition coding, binning, coded time-sharing and rate-splitting 





superposition 


binning 


rate-splitting 


coded time- sharing 


reference 


T~- — i u i 

multiple access channel 


/ 

V 






/ 

V 


1 1/|, 1 IDJ 


more capable broadcast channel 


/ 

V 








1 1 / 1, 1 ISJ 


semi -deterministic broadcast channel 




/ 

V 






1 1 Q 1 

1 1 


broadcast channel, degraded message set 


/ 








|20| 


Gaussian MIMO broadcast channel 




/ 






|21| 


interference channel, strong interference regime 


/ 




/ 


/ 


I15J, UU 


deterministic interference channel 


/ 




/ 


/ 


[22] 


cognitive interference channel, better cognitive decoding 


/ 




/ 




[231 


semi-deterministic cognitive interference channel 




/ 


/ 




|24| 


Gaussian cognitive interference channel, weak interference 




/ 






1251 


Cognitive interference channel, degraded message set 


/ 




/ 




|26| 



A. Main Contributions 
In this paper: 

1) We propose a novel chain graph representation for encoding schemes based on standard random coding tech- 
niques. This representation is for a general, single-hop, memoryless, multi-terminal network used without feed- 
back/cooperation where the number of transmitters and receivers, as well as the set of messages amongst 
transmitters, is arbitrary. This new formaHsm provides a clear and unified framework to represent achievabiHty schemes 
based on random coding arguments; it includes and generalizes all known achievable schemes for the class of networks 
considered. 

2) We derive the achievable rate regions based on the proposed chain graph representation. The proposed chain graph 
representation naturally leads to the generalization of the error analysis using classic bounds of the error probability and 
typicality arguments such as the covering lemma and the packing lemma. The achievable rate regions are obtained by 
the analysis of the chain graph representation of the achievable scheme for any channel in the class we consider 

3) We compare the rate regions obtained using different bounds for the probability of encoding and decoding 
errors. We explicitly characterize the rate bounds obtained using the covering lemma and mutual covering lemma to 
bound the encoding error probability, and sequential decoding and joint decoding to bound the decoding error probability. 
By comparing the corresponding achievable region, it is possible to determine how these different bounding techniques 
compare for a general channel. 

4) We re-derive a few achievable regions for classic channel models to provide simple and yet insightful examples of 
the chain graph representation. We focus on the multi-access channel, the broadcast channel, the interference channel, 
and the cognitive interference channel and re-obtain the largest know achievable regions for these channels as well as 
introducing new transmission strategies. 

B. Paper Organization 

Section presents the class of networks considered in this work and revises the standard random coding techniques that we 
employ in our general achievable scheme. Section|III]introduces the novel chain graph representation of the encoding operations. 
Section |IV] describes the codebook generation, encoding and decoding procedures of the achievable scheme associated to a 
specific graph representation. Section |V] derives the rate bounds that define the achievable rate region based on the proposed 
chain graph representation. In Section |VT] we derive a series of achievable schemes for classic information theoretical models. 
Section fVU\ concludes the paper. 
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II. Channel Model and Random Coding Techniques for Achievability 

A. Notation 

In Sec. II we define a general single-hop, memoryless, multi-terminal network without feedback/cooperation. Given the 
generality of this model we try to adopt a notation that is clear, yet intuitive, when describing general subsets of users, 
messages, and random variables (RV). We consider a channel with -/Vxx transmitters and N^x receivers. In order to designate 
arbitrary subsets of transmitter and receiver nodes we use bold lower case letters: 

iC^AT -q3({l...iV}), (1) 

where *p indicates the power set and N is either A^tx or N^x- 

To compactly index the messages from certain transmitting nodes to other receiving nodes, we use bold upper case letters 

SC^N, x^N, =^i[l...Ni])xVp{[l...N2]), (2) 

where generally we will have {Ni,N2) = {Ntx, N^x)- When no conditions are specified on a set operation, the operation is 
assumed to be over all the elements in the set, i.e. 

^ai= ^ ai, (3a) 

i i, V iCVn 

^as= ^ as. (3b) 

s s, V scq3«i.«2 



Also let 



where 



^ a; = J2 «i' (4) 



= {i e ^Pat s.t. condition (i) holds for 1} 



We adopt the following conventions for superscripts and subscripts: 

• index k/z: transmitters/receivers, for fee [1 . . . A'rx] and z e [1 . . . A^rx], 

• Xf^jY^: channel inputs/outputs at transmitter k and receiver z, 

• index i/j: subset of transmitters/receivers. We will also use 1/m and v/t when dealing with multiple subsets, 

• index S: set of containing (i,j) pairs, 

• S: complement of the set S in ^at^x ^ *P^Vrx- 

• [iVi . . . A^2] indicates the subset of N between iVi and N2. 

B. Properties of typical sequences 

The majority of the results in the paper are derived using properties of letter-typical sequences; we adopt the definition of 
letter-typicality in ||2ll, ||29ll. Il3l, l|4l. 

Definition Let be a finite sequence whose ith entry takes values in X and let iV(a|a;^) be the number of occurrences of 
the letter a in for a £ X where X. For e > 0, we say that a sequence x^ is e-letter typical with respect to Px if 

^{a\x'') - Px{a) <e-Px{a) V a G X. (5) 

The set of x^ satisfying dll-BI l is called e-letter typical set (or, more simply, typical set) and is denoted as 'J^(Px). The 
letter typical sequences are those sequences whose empirical distribution is close to Px- The next theorem establishes the 
fundamental properties of typical sequences. 

Tlieorem II.l. Suppose < e < /ix. G cind distributed as i.i.d. draws from the distribution P^, then the 
following holds 

(1 - <5,(7V))2^(i-^)^(^) < |Tf I < 2^(i+^)^(^) (6b) 
l-^e(iV) <P[X^ eTf(Px)] < 1, (6c) 

for some Se{N) such that Se{N) as N 00. 

The properties of typical sequences in Th. III. II are crucial in deriving transmission schemes based on random coding. 
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TABLE II 
The messages for a general IPC. 





from Txl from Tx2 from both Tx's 


to Rxl 
to Rx2 
to both Rx's 


Wl^2 W2^2 VF{i_2}-2 
W'l^{l,2> M^2^{1,2> M^{l,2|-»fl,2} 



C. Network Model 

We consider a multi-terminal network where A'tx transmitting nodes want to communicate with iV^x receiving nodes. A 
given node may only be a transmitting or a receiving node (and may not alternate between them), that is, the network is single- 
hop and it is used without feedback/cooperation. The transmitting node k, k E [1 . . . A^tx], inputs Xk to the channel, while 
the receiving node z, z G [1 . . .-/Vrx], has access to the channel output Yz- The channel transition probability is indicated 
with Pyi...Yn^^\Xi...Xn^^- The channel is assumed to be memoryless. The subset of transmitting nodes i, i G *Pwtx' is 
interested in sending the message W^i^j to the subset of receiving nodes j G ^at^x o^^r ^ channel uses. The total number 
of messages is (2^"^^ — 1)(2^'^^ — 1) and includes all possible combinations of "degraded message sets'Vcognition. The 
message (i, j) G ^Ntx ^ ^Pa^rx' uniformly distributed in the interval [0 . . . 2^-'^'^j — 1], where N is the block-length 

and Ri-,j the transmission rate. The outcome of the RV Wi^j is denoted with Wi^^ and the set of all messages is denoted by 
w = {wi^i, V (i, j) G <PjVTx X *PjVrx}- 

A rate vector R — V (i, j) G ^Ntx ^ ^n„k} is said to be achievable if there exists a sequence of encoding functions 

= , {{Wi-.i, s.t. (ij) c Vn^^ X qj^^x, ft G i}) , 



and a sequence of decoding functions 



such that 



Wi'^^ = Wt^:(Y,^] if zej, (7) 



lim max I 



The capacity region C(R) is the convex closure of the region of all achievable rates in the vector R-pairs. 

Determining capacity is, in general, a complex task that requires an extensive understanding of the channel model, for this 
reason one is often interested in determining a subset of the capacity region where some rates have been fixed to zero. Let 

e(RT), T c ^Pat^x X *Pjvhx (8) 

denote the capacity region restricted to the plane Rt — {Ri-^j, V (i, j) G T} which corresponds to the capacity region of 
a less general channel where the rates (i,j) G T} have been set to zero. 

Fig. [T] shows the channel model considered in this work: on the left of the figure there are the A^tx transmitting nodes, 
each with input Xk to the channel. The channel is described by the transition probability Pyi...Yn-^^\Xi...Xn^^ ^nd, given the 
channel inputs [Xi . . . ^Af^x] randomly produces the channel outputs [Yi . . . i^TVnx] which are then received at Njix decoders 
on the right. 

A large number of channel models studied in information theory are included in the class of networks, such as the multiple 
access channel, 1121 . lfT6l . the broadcast channel ll30l . Q, lH, the interference channel 1311 . Il32l . ifTl I. and the cognitive 
interference channel |l33l, llMl, lUS- 



D. An example: the general two-user interference channel 

To make the reader familiar with the notation in Sec. III-AI and the general model in Sec. III-CI we describe the general 
two-user interference channel (general IFC), that is the model in Sec. III-CI in the case of A^tx = X-rx — 2. 

This channel model is depicted in Fig. 121 two transmitters communicate to two receivers through the channel Pyi.Y2\Xx,X2- 
The total number of messages to be sent over the channel is (2^ — 1)(2^ — 1) = 9 and is obtained by considering all the 
possible ways in which a message can be encoded by any subset of transmitters and decoded by any subset of receivers. 

All the nine possible messages are listed in Tab. [Ill using the notation in (|2|i: Wi^j and W2^j are the messages known only 
at transmitter 1 and 2, respectively, while message W{i 2}^j is a message known at both. Similarly, Wi^i and Wi^2 are the 
messages to be decoded only at receivers 1 and 2, respectively, while message VFi^^i 2} is to be decoded at both. 

In literature, one often refers to the message Wi^i as a private message, W{i,2}-»i as a common message and to Wi^{i.2} 
as a degraded message. It is clear from this simple example that this nomenclature is somewhat confusing when considering the 
most general distribution of messages even for a channel with small number of users. For this reason we avoid this terminology 
and choose instead the notation in ([T]l and in (|2|. 
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Channel Inputs 



Channel Outputs 



Set 1 of 
transmitting 
nodes 

— K 



r 




y 



l^J 



Set J of 
receiving 
nodes 



Fig. 1. The general cognitive multi-terminal network. 



TABLE III 

Specific subcases of the general interference channel 



subcase 


channel model 




point-to-point 


e({Ri^i,i?2^i}) 


MAC 


e({i?i^i,/?2-i,R{i,2}^i}) 


MAC with common message 


e({i?i_i,iJi^2}) 


BC 


e({R{1.2Hl,^?{l,2}^2}) 


BC 


e({Ri^i,/?i^2,i?i^{i,2}}) 


BC with common message 


e({Ri^i,i?2-.2}) 


IFC 


e({i?l^l,i?2-2,R{l,2}^{l,2}}) 


IPC with common message 


e({Ri-.i,/?{i,2}-.2}) 


CIFC 


e({Ri-i = ^{i,2H{i.2}}) 


CIFC with degraded message set 


e({/?l^(l,2|,fi2^fl,2}}) 


compound MAC 



The general IFC contains a number of classic information theoretical models as a special case: Tab. Hill lists all subcases of 
the general IFC that have been studied in literature using the notation in (H). Note there are a total of 2^ subcases of the general 
IFC. Of these 1? subcases, capacity is known for the 2^ cases in which only one message is being transmitted. Capacity is 
also known for the MAC, MAC with common information, and the compound MAC. In all the other subcases capacity is not 
known in general. 

It is interesting to note that, despite the simplicity of this setup, some of the subcases of the general IFC have never 
been considered in literature. For instance the capacity C({i?i^i, i?{i.2}->2}) has never been investigated, although this 
communication problem is relevant in the study of cognitive networks. We will return to the models in Tab. [Ill] in Sec. 
I VI I to derive achievable regions for some of these models using a new general achievable scheme that is the topic of this paper 

E. Random Coding Techniques for Achievability 

We revise here standard random coding techniques used in the literature for achievability in single-hop networks used without 
feedback/cooperation: coded-time-sharing, rate-splitting, superposition coding, and binning. 

• Coded Time-Sharing consists of using different transmission strategies according to a random schedule fTT) and allows 
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Encoded Messages 



Decoded Messages 



W^1^{1,2} 



Network Model 



^"{1,2}^! 
W^{1,2}^2 
^{1,2}^{1,2} 



TX 1 V X^ 




H^2^1 
W^2^2 
W^2-*{1,2} 



Tx 2 ^ X 



Rx2 



H^2^1 
W^{1,2}^1 



M^1^{1,2} 
W^2^{1,2} 
W^{1,2}^{1,2} 



H^1^2 
iy2^2 
W^{1,2}^2 



Fig. 2. The general IFC with the most general set of messages amongst transmitters. 



one to achieve the convex closure of the set of achievable points and, in some cases, an even larger region ||2l pp. 288-290]. 
Coded time-sharing also includes time multiple access (naive time-sharing) and frequency multiple access as special cases. 
Let denote a random instance of the coded time-sharing sequence generated according to the distribution 

N 

and let be revealed to all the encoders and decoders in the network. In coded time-sharing the users generate multiple 
random codebooks conditioned on the value of the sequence q" and choose their transmission strategy according to this random 
outcome. 

Coded time-sharing outperforms naive time-sharing and consists of using different transmission schemes for different fractions 
of time. As before, let denote a random instance of the time-sharing sequence generated according to the distribution 
in (|9|l, and let q^ be revealed to all the encoders and decoders in the network. Moreover let Rj denote the rate achievable 
under the strategy j. In naive time-sharing the achievable strategy j is used at time i qi = j: with this time-sharing strategy 
it is possible to achieve the region 

R = ^p[g-j]R,. 

Taking the union over all the possible distribution of Q, naive time-sharing achieves the convex closure of the achievable 
regions Rq. 

• Rate-Splitting corresponds to dividing a message Wi^j into a set of sub-messages {Wi'^j/} where j C j' and i D i', 
that is, in sub-messages that are encoded by a smaller subset of transmitters and decoded at a larger set of receivers than 
the original message. Increasing the number of messages to be transmitted over the channel also increases the number of 
transmission strategies available at the encoders 111 11 . This can potentially result in a larger achievable region with respect to 
the original problem, as it makes it possible to implement more complex communication strategies. 

Let wl'^j^ indicate that sub-message Wi^m obtained by splitting the message Wi^j, then the message Wi^j can be split 
in a sequence of sub-messages W^i'lTn ^'^^ every (1, m) such that 

j C m and i D 1. (10) 
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The sub-messages W^^^ are uniformly distributed over the interval [0 . . . 2^^i^m — l] so that 



for 



(l,m), (To) (l,m), (To) 



7j!ri = §f^, E 7lri! = i, ai) 



(l,m), (To) 



where is the rate of the RV Wi^j in the original channel and in the rate of the RV Wi^j in the channel after 
rate-splitting has been applied. 

Rate-splitting effectively transforms the problem of achieving a rate vector R into the problem of achieving the rate vector 
R' such that 



R' = TR, (12) 



and where the matrix F in dTTT l is defined as 



(i.j)x(l,m) = 7l- 



Rate-splitting is useful in the cases where it allows to increase the number of messages in the channel, thus effectively 
increasing the viable transmission strategies. Rate-splitting is sometimes also useful for those channels in which it does 
not increase the messages to be transmitted but for a different reason. Consider the BC with common information of |35|, 
C({i?i_»{i_2}, ^1^1, ^^1^2}), and assume that the rate vector R is achievable. From the achievability of R one also deduces 
that the rate vector 

R = (q;o-Ri^{i,2}, ^1^1 + ai-Ri^{i,2}, -Ri^2 + a2-Ri^{i,2}) (13) 

is achievable for any aQ,ai and a2 such that ao + ai + a2 = 1. This is because parts of the messages Wi^i and Wi^2, 
decoded only at decoder 1 and 2 respectively, can be encoded with the message VFi^^i 2} which is decoded at both receivers. 
With this consideration one can increase the achievable region of a given scheme, especially in those scenarios where the rate 
of the message Wi^i is limited by the interference created by the message Wi^2 and vice versa. The operation in ( fT3]) is a 
form of time-sharing but, depending on the complexity of the achievable region, it is sometimes easier to interpret it as an 
operation that enlarges the overall achievable region, regardless of the specific achievable scheme involved. 

• Superposition Coding can be intuitively thought of as stacking codewords on top of each other fSO). The "base" codewords 
are decoded first and stripped from the received signal so as to reduce the interference when decoding the "top" codewords. 
In superposition coding a different top codebook is generated for each base codeword and the codewords in the top codebook 
have a conditional distribution that depends on the specific base codeword. Since top codewords are generated according to a 
probability distribution conditioned on the base codeword, superposition coding increases the error performance of the code. 
If the top codewords were created independently from the base codewords, the joint distribution among each base codeword 
and each top codeword would not change with a decoding error, thus resulting in more errors. 

Superposition coding is useful in broadcasting two messages to two sets of receivers j and m where m C j by having 
the codeword for the set j superposed to the codeword for the set m. The users in the set j only decode the base codeword 
while the users in the set m decode the base codeword first and, using the knowledge of the base codeword, decode the top 
codeword. 

Let U\^rn be the RV with distribution Pui^^ carrying the message W{^^ and the RV J/i^j carrying the message W(^^ be 
superposed to the RV J/i^m- A different top codebook is associated to each base codeword C//^j — and the codewords 
in each of these codebooks are generated according to the distribution ^'(7i^j|(7i^„=ui_m- 
Superposition of one RV Ui^j over another RV C/i^m can be performed when the following two conditions hold: 

• 1 C i: that is the bottom message is encoded by a larger set of encoders than the top message. 

• m C j: that is the bottom message is decoded by a larger set of decoders than the top message. 

If C/i^j is superposed to C/i^m and Uv^t is superposed to C/i^m, than Ui^j is also superposed to C/v^t- Similarly, if Ui^j is 
superposed to C/i^m, then the reverse cannot hold. The binary relation "A is superposed to B" therefore establishes a partial 
ordering amongst the encoding RVs denoted by A<B. 

• Binning is sometimes referred to as Cover's random binning p) Gel'fand-Pinsker coding 131, and allows a transmitter to 
"pre-cancel" (portions of) the interference known to be experienced at a receiver This is done by generating a codebook that 
has a larger number of codewords than the rate of the associated message so that the transmitted codeword can be chosen 
according to the specific value of the interference. With the appropriate choice of conditional distribution between the message 
and the interference, the encoder can reduce the effect of the interference at the intended decoder. Although in the original 
setting of JS) the interference is a random process, this technique can be extended to the case where the interference is a 
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different user's codewords. In the Gaussian channels (30] and deterministic or semi-deterministic channels |fT9l . binning can 
completely pre-cancel the effect of the interference on the channel output. In the Gel'fand-Pinsker problem this result is known 
as Costa's "writing on dirty paper" [36|. 

Binning is useful in broadcasting two messages to two sets of receivers j and m where m n j = by having the codeword 
for the set of receivers m binned against the codewords for receivers j or vice versa. 

Let again Ui^m be the RV with distribution Pui^^ carrying the message Wl^^ and the RV Ui^j carrying the message W^'_^j 
be binned against the RV t/i^m- The codewords U(^-^ are generated independently from the codewords but chosen so to 

look as if generated according to the distribution ^c/i^j|;7i^„- In order to find a codeword [//^j that appears to have the desired 
marginal distribution, it is necessary to produce more codewords than the rate of message Wj^j, and the number of 

excess codewords depends on the joint distribution ^Vi^j.d^m- The RV JJi^j can be binned against the RV U\^m when 

• i C 1: that is, the set of encoders performing binning has knowledge of the interfering codeword 

Note that if [/i^j can be binned against C/i^m, then C/i_,j can also be binned against Ui^rn, regardless of the value of j and 
m. This is commonly referred to as "joint binning" |37l. The relationship "A can be binned against B" is transitive and 
symmetric for the RV known at the same set of decoders. In the following "binning Ui^j against the RV U\^rn" is indicated as 
C/i-»j <C/i^m- For ease of notation, we introduce the following short hand notation for joint binning: AoB A<B, B<A. 

By combining these four encoding techniques one can design an achievability scheme with specific characteristics. Although 
the code design detailed above is fully general, some coding choices result in an ill-defined construction. Consider for instance 
the code 

In this case it is not possible to define a joint probability distribution for Ui^i, Ui^2, C^i^{i,2} because of the cyclic dependency 
between the RVs. We address this issue in the next section. 

III. The Chain Graph Representation of a General Achievable Schemes 

The elements included in the random coding construction of Sec. III-EI mav be compactly represented using the following 
graph 9{V,E) : 

• every vertex v — E V C *P(A'tx) x ^(^rx) of the S is associated to the RV C/j^j carrying the message W(^^ at 
rate obtained through the rate-splitting matrix F from rate vector R as in ( fTTI ). 

• the set of edges E connecting the vertices in V contains two subsets S and B such that E — SU B and S n B = 

• the vertex C/i^j is connected with U\^in by a directed edge of type S (for superposition), if [/i^rn~C^i-»j (solid line). 

• the vertex Ui-^j is connected with Uv~>t by a directed edge of type B (for binning), if C/v^t < Ui^j (dotted line). 
Since ~ is a transitive relationship, it is convenient to omit the edges of type S that are implied by transitivity when 

representing S- The coded time-sharing RV Q is not represented in this graph as it is assumed that each RV is generated 
according to a marginal distribution that depends on Q; the overall achievable region may then be obtained considering the 
union over all the possible distributions of Q. 

The graph representation of the achievable scheme is particularly useful in deriving the joint distribution of the coding RVs 
Ui^j in a general scheme involving superposition coding and binning as described in Sec. |ll] When trying to determine this 
distribution, superposition coding and binning effectively result in allowing for any joint distribution among the connected 
RVs. In the following we detail how the graph 9{V, E) can be used to describe the dependency structure of the codewords of 
an achievable scheme for a general multi-terminal network. Graphs representing conditional dependencies among RVs have 
been extensively studied in the literature in the field of graphical Markov models ll38l . |[39l . Il27l . In the following we briefly 
introduce three graphical Markov models and describe how this modeling tool can be used to describe the joint distribution 
among codewords in a general achievable scheme. 

A. Graphical Markov Models 

In graphical Markov models, the vertices V of the graph 9{V, E) represents a set of RVs while an edge between two vertices 
indicates the absence of conditional independence among the two RVs. More formally, given a graph 9{V,E), a graphical 
Markov model is obtained by defining a joint probability distribution P{V) on the product probability space 

"X — X -^c/fci 

such that P{V) is global G-markovian, that is 

A,BeV, ALB\ S[P], (14) 

for any set of nodes S that separates A and B. 0. 

^Other definitions of Marlcov model are possible. We present only this definition for the sake of space. See 1271 for a more complete introduction to 
graphical Markov models. 
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Not every graph can be used to define a graphical Markov model: among the suitable graphs, three classes commonly studied 
are: 

• UDG: An UnDirected Graph is a graph were all the edges are undirected. 

• ADG: An Acyclic DiGraph is a graph were all the edges are directed and there is no directed cycl^ 

• Chain graphs: A chain graph is a graph with both directed and undirected edges but that contains no directed cycles. 
ADGs provide a convenient and recursive factorization of the probability distribution P{V) using parent nodes which allows 

one to calculate the joint distribution of any subset of nodes of V. ADGs do not uniquely represent P{V), that is, multiple 
ADGs can describe the same joint probability distribution. ADGs are often used in Bayesian inference to represent causal 
relationships by having the edges represent a causal relationship the connected RVs. 

Unlike ADGs, UDGs generally do not allow a simple factorization of the probability distributions P{V). On the other hand, 
a UDG is uniquely determined by its associated Markov models. A convenient factorization of the joint distribution for UDGs 
is available for the class of decomposable UDGs, where the joint distribution can be factorized over the cliques of the graph. 

Chain graphs encompass both ADG and UDG and therefore are a more general modeling tool. Chain graphs are also an 
important tool with which to understand the relationship between the two models: ADG and UDG both represent a joint 
probability distribution P{V) and it is not always clear when a UDG is equivalent to an ADG, that is when they represent the 
same joint probability distribution. Studying the equivalence among chain graphs is useful in understanding the equivalence 
between the two specific subclasses. 

Unfortunately they do not allow for a general factorization of the joint distribution P{V) as ADGs and are not uniquely 
associated to a Markov model as UDGs. Despite this, chain graphs are useful when studying the relationship between different 
models: by establishing when an ADG or a UDG is equivalent to chain graphs, one can understand the underlying relationship 
between Markov models and their graph representation. 

The graph representation of a general achievable region presented at the beginning of this section corresponds to a chain 
graph. Superposition coding and binning both coiTesponds to directed edges that indicate that the distribution of U\^^ depends 
on Ui^j for both U\^-w<Ui^j or U\^vr < f^i^j- Undirected edges can be obtained in one of two ways: 

• joint binning: Ui^j < t/i^m and Ui^m < Ui^j 

• superposition coding and binning in the reverse direction ?7i^j~?7i^m and C/i^m < C/i^j 

A schematic representation of the graphical Markov model associated with the graph Si{V,E) is presented in Fig. |3] each 
vertex (i, j) is associated with a corresponding message Wf^^, a rate-splitting equation and an auxiliary RV Ui^^ with distribution 
P[/. jlQ according to which the codebook is generated. The vertex (i, j) can be connected to a vertex (1, m) by two types of 
edges: a superposition edge S and a binning edge B; both edges indicate the Markov dependency between [/i^j and Ux^m 
given Q. 




istribution 

[/v^t|[/i^j, 



S : superposition edge 
B binning edge 



IVIessage 
Distribution 

Rate splitting 

-^Uj=Z](l,m), (7) TlLm-^i^ 




Distribution 



Fig. 3. A schematic representation of the graph in Sec. IIIII . 



^Sometimes also referred as Acyclic Directed Graph (DAG). We adopt this nomenclature from 1271 . 
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^{1,2}-{1,2} 



Fig. 4. The graph representation of an achievable scheme in Section Ull-A 1 1 based on superposition coding for the CIFC. 



1 ) Graph Representation of an Achievable Schemes based on Superposition Coding: The Markov graph representing an 
achievable scheme employing only superposition coding, rate-splitting, and coded time-sharing, is an ADG. The superposition 
step corresponds to a direct edge in the graph representation and since the relationship ~ is a partial order relationship, no 
cycles or undirected edges can occur in the graph. 

For an ADG it is possible to write the factorization of the joint distribution using the parent nodes, that is. 



(15) 



where pa(Uk) indicates the parent nodes of Uk, that is the set of all the nodes connected to Uk by a direct edge. 

This implies that the joint distribution associated with the graph representation of the general achievable scheme can be 
written as; 



Pm^j. V (i.j)},Q - n Pu;^j\ {Ui^rr., (7i^j<C/i^„},Q' 
(i-j) 

for any rate-splitting matrix F and coded time-sharing RV Q. 



(16) 



As an example of an achievable scheme based on superposition coding, consider the CIFC, C i?{i,2}-»i}) with the 

following rate-splitting matrix: 



P'l^i 

P'l->2 
Pl->{1,2} 

P'2^2 
R 



{1.2}^2 



'-{1,2}^{1,2} 

and the achievable scheme involving only superposition: 



[1^1] 

[1^1] 
7L2 
[1^1] 

7l-.{l,2} 








11^2 



J{h2}^2] 
'{1.2}^2 
J{1.2}-2] 
7{1,2}^{1,2} 



^1^1 
^{1,2}^2 



(17) 



'^{l,2}^2<t^{l,2}^{l,2}j 
C/2^2<t^{l,2}^2i 
C/i^2<t^{l,2}^2, 
C/l^l<C/l^{l,2}. 



(18a) 
(18b) 
(18c) 
(18d) 



The achievable scheme in (fTSl l corresponds to the graph representation in Fig. lU the solid lines represent the 5 edges 
associated with superposition coding but the edges that are implied by the transitivity of the relationship ~ have been omitted 
from the plot. The blue, rhomboidal nodes carry a rate-split of the Wi-ti message while the green, square ones of the message 

W{i,2}^2- 
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From the graph representation of the achievable scheme in Fig. |4]it is possible to quickly derive the joint distribution of the 

RVs Ui^j as 

'^{t^{l,2}-.{l,2},t'"{l,2}-.lC/{l,2}-.2f l^l'^1^2'72^2} (19a) 

^c/n,2}-.n.2} (19b) 

-fc/l^{l,2}l'7{i,2}^{l,2} (19c) 
-PC/{1,2}^2|'7{1,2}~,{1,2} (19d) 

-Pc/i^i|c/i^{i,2},(7{i,2}^{i,2} (19e) 

^C/2^2|C/{i,2}^2,t'{l,2}^{l,2} (19f) 
PUl^2\Ui^^l 2},U{1^2}^2,U{1.2}^{l,2} ■ (19g) 

2) Graph Representation of an Achievable Schemes based on Joint Binning: The Markov graph representation of an 
achievable scheme employing only joint binning, rate-splitting, and coded time-sharing is a UDG since the joint binning 
corresponds to an undirected edge in the graph representation of the achievable scheme. 

Unfortunately it is not possible to provide a convenient factorization of the joint probability distribution of a general UDG 
but only for the class of decomposable UDG. A UDG is decomposable when it can be rewritten as a series of maximal cliques 
C, connected by separator sets Sj. A simple way to establish whether a UDG is decomposable is by checking whether it 
is chordal. An UDG is chordal if every cycle of length greater than 3 possesses a chord, that two non consecutive nodes 
that are neighbors. Every chordal UDG is also decomposable and vice versa, i.e. the two conditions are equivalent. The joint 
distribution of a Markov model of a decomposable UDG can be expressed as 

P{V) = ^c^^cP{u.,u..c.}^ (20) 

where C is the set of the maximal cliques and 5* is the set of separators. Another interesting property of decomposable UDG 
is that they are equivalent to some ADG obtained by choosing the appropriate direction for each edge of the original UDG 
||271 . With this consideration we can rewrite the joint distribution in ( |20] | using the factorization in (fTsT i for the equivalent 
ADG. The direction of the edges in the equivalent ADG are obtained using the moral graph of the original UDG: we will not 
introduce this concept here as it is outside the scope of the paper, instead we direct the interested reader to ll39l . 

In general, the graph representation of an achievable scheme involving joint binning in not chordal nonetheless such an 
achievable scheme can be made chordal by adding a further joint binning step. Consider an achievable scheme where having 
a cycle of length n > 3 and lacks a chord: 

Ui^i, < >Ui^i, ■ ■ ■< >[/i^„_, < >C/i^„. (21) 

This scheme can be made chordal by adding the joint binning step 

C/i^, <>;7i^j3. (22) 

This additional encoding step can only enlarge the achievable region, since the original scheme can be reobtained with the 
appropriate choice of the distribution imposed by the joint binning between C/i^ji and Ui^j^. 

For this reason, in the remainder of the paper, we will always assume that all the undirected cycles are chordal by assuming 
that the joint binning relationship <> is transitive, that is 

Assumption 1. For all [/j^j < >C/i^m the following holds 

Ui^jOUi^^, Ui^rnOUi^j =^ C/™<>t/i^t. (23) 

With Assumption [T] the joint distribution associated with the graph representation of the general achievable scheme can be 
written as: 

P{U;^i, V (ij)},Q = n Pu,^i\ c/i^j^c/,^„},Q' (24) 

(ij) 

for any rate-splitting matrix F and coded time-sharing RV Q, where <i is the set of directed edges obtained by converting the 
undirected edges of the decomposable UDG in the directed edges of the equivalent ADG. 

As an example of achievable scheme based on joint binning, consider again the CIFC, C ({i?i^i, i?{i,2}^i}) ^ with the 
rate-splitting matrix F in JTtI i and the achievable scheme employing joint binning as follows 

C^i^{i,2} < >C^i^i < >Ui^2 (25a) 

C^{1,2}^{1,2} < >f^{1.2}^2- (25b) 
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K J 












^{1,2}- 



^{1,2}-{1,2} 



Fig. 5. The graph representation of an achievable scheme in Sec. IIII-A21 based on joint binning for the CIFC. 



The achievable scheme in dZSl l corresponds to the graph representation in Fig. |5] The dashed lines correspond to the joint 
binning relationship <> while the dotted lines correspond to the equivalent ADG. 

From the graph representation of the achievable scheme in Fig. |5]it is possible to quickly derive the joint distribution of the 
RVs as 



{'^{l,2}-.{l,2}.C^{l,2}-.lt^{l,2}^2C^l-»l'^l-2t'2^2} 

Pi 



■'{1.2}^{1.2} 



{1,2}^2|'^{1,2}^{1,2} 



Pi 



p 

PUl^l\Ul^2Ui^^l_2} 
PU2^2 ■ 



(26a) 
(26b) 
(26c) 
(26d) 
(26e) 
(26f) 
(26g) 



3) Graph Representation of an Achievable Schemes based on Superposition Coding and Binning: After having considered 
achievable schemes based only on superposition coding, which correspond to an ADG, and some based only on joint binning, 
which correspond to UDG, we turn our attention to the general case by considering both binning and superposition coding. 
A chain graph is a graph with both directed and undirected edges and does not contain any directed cycle. To show that the 
graph representation of a general achievable scheme can indeed be modeled using a chain graph Markov model, no directed 
cycle can exist in the graph representation. This condition can be obtained with the following assumption 

Assumption 2. RVs known at same set of decoders i form do not form directed cycles in the graph representation. If a cycle 
exists it must be undirected. 

Given any achievable scheme, one can always obtain an achievable scheme that satisfies Assumption |2]by adding the binning 
edges that transform a directed cycle into an indirected cycle. This additional encoding operation can only enlarge the achievable 
region since the original scheme can be reobtained with the appropriate distribution imposed by the binning step. 

Among the class of chain graphs we are particularly interested in is the class that has a Markov equivalent to some ADG, 
so that we can employ a recursive factorization of the joint probability distribution as in (fTSl l. A general achievable scheme 
corresponds to a chain graph that is ADG equivalent when Assumption [T| holds together with the following assumption. 

Assumption 3. RVs that are jointly binned, have the same parent nodes, that is, if Ui^^oUi^t and C/i^m<C^i->j or C/i^m~t/i^j, 
than Ui^m < Ui^t or Ui^m<Ui^t- 

As for Assumption |2] given any achievable scheme, it is possible to add further binning edges for Assumption [3] to hold 
and by doing so one can only obtain a larger achievable scheme. 

We summarize the necessary assumptions for a graph representation of a general achievable region to correspond to a chain 
graph that is equivalent so some ADG in the next theorem. 

Theorem III.l. If Assumptions \l] |2] and\3\ hold, the graph representation of a general achievable scheme corresponds to 
chain graph which is equivalent to some (non-necessarily unique) ADG. 
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t^i^{i,2}L^::::^{i,2}-2 

\ z 



f^{l,2}^{l,2} 



Fig. 6. The graph representation of an achievable scheme in Section IIII-A3l based on binning and superposition coding for the CIFC. 



If an achievable scheme does not satisfy these assumptions, it is possible to add some additional binning edges and obtain 
a scheme that satisfy such assumptions and has a larger achievable regions. 

Proof: Since the proof involves some graph theoretic considerations that are outside the scope of the paper, the complete 
proof is relegated to App. |A] 

The main point of the proof is in showing that under the given assumption, joint binning forms connected components, or 
chques, that are well connected to the rest of the graph, so that one can pick a direction for these undirect edges and form an 
equivalent ADG. 



We refer to a graph representation of an achievable scheme in Th. IIII.ll as a Chain Graph Representation of an Achievable 
Scheme (CGRAS). 

A modified Maximum Cardinality Search (MCS) algorithm can be used to efficiently construct a Markov equivalent ADG 
of a CGRAS [27 1. Since the edges of the equivalent ADG are ^nerated by orienting undirected edge, the equivalent ADG 
can be written as S(V,B- U 5) for some B- C B. The notation < is used in the following to indicate the edges in B-. With 
this notation, the joint distribution of a CGRAS can then be factorized as: 

n^!7i^j|{(7i- 
(iJ) 



Pi 



{(7i^j, V (ij)} 



;7i^j<;7i^„ or Ui^i^Ui^^}- 



(27) 



As an example of a CGRAS, consider again the CIFC, C ({i?i^i, i?{i,2}^i})' with the rate-splitting matrix F in ([TT) and 
the achievable scheme employing superposition coding: 

C^{i,2}^2<t^{i,2}^{i,2}, (28a) 

C/2^2<t/{l,2}^2, (28b) 

Ui^2<U{i^2}^2, (28c) 

i7i^i<t/i^{i,2}, (28d) 

and binning: 

C/i^{i,2}<C/{i,2}^2, (29a) 

C/i^i<C/{i,2}^2, (29b) 

C/i^i < >Ui^2- (29c) 

The graph representation of this achievable region is in Fig.|6l as for Fig. |4] and Fig.|5] the solid lines represent the 5 edges, 
the dashed lines the B edges and the dotted lines indicate the direction in B- to be associated with the undirected edges. The 
solid edges that are implied by the transitivity of the ~ have been omitted from the plot. The blue, rhomboidal nodes carry a 
rate-split of the Wi^i message while the green, square ones of the message W3,2- 
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From the graph representation of the achievable scheme in Fig. |6]it is possible to quickly derive the joint distribution of the 

RVs Ui^j as 

'^f^{l,2}-^{l,2> T'^{l,2}-*2:f^l-*{l,2} T'^l^l5f^l->2 ,C^2-*2 (30a.) 

Pu^,,.,^^,.., (30b) 

^'^{1.2}-2|'7{1.2}^{1,2} (30c) 

^'7i^{i,2}|(7{i,2}^{i,2} (30d) 

-f'72^2|C/{l,2}^2,'7{i,2}^{l,2} (30e) 

-Pc/i^2|;7i^{i,2},c/{i,2}^2,;7{i,2}-.{i,2} (30f) 

-Pc/l^l|;7l^2,C/{l,2}^2,C/i^{l,2},(7{i.2}^{l,2}- (30g) 

The achievable scheme in Fig|6]is a more general scheme than the largest known achievable region for the CIFC in f23l. 
The scheme we propose here has one more RV than the scheme in |,23J - 1/2^2- This RV carries a private message between 
transmitter 2 and receiver 2 that is not encoded at the cognitive encoder. This added RV is considered mostly for illustrative 
purposes: since the cognitive transmitter cooperates in the transmission of the primary receiver in transmitting [/{i,2}^2' one 
does not expect to have any particular advantage in adding to the achievable scheme in [23 J. 

At decoding, each receiver z decodes a subset of the CGRAS from the channel output z. The structure of the codebook 
observed at decoder z relates to the subgraph S{V^,S^ UB^) where 

^"-{(iJ)eV^, ^ej} (31a) 
= {(i, j) X (1, m) e 5, z e j, z e m, } (31b) 
= {(i, j) X (1, m) e B, zej, z e m, } . (31c) 

We refer to the S{V^,S^ UB^) as the graph representation of a general achievable scheme at decoder z. We would like 
the graph representation of a general achievable scheme at decoder z to have the same convenient factorization of the joint 
distribution as the original graph. The next theorem proves that this is indeed the case. 

Theorem III.2. For every receiver z, if Assumptions\l\ |2] and\3\ hold, the graph representation of a general achievable scheme 
at decoder z corresponds to chain graph which is equivalent to some (non-necessarily unique) ADG. 

Proof: The proof is provided in App. |B] ■ 
If g(y^SUB) is a CGRAS, we refer to g(V^^5^ UB^) as the CGRAS at decoder z. 



For the CGRAS in (30gl the CGRAS at decoder 1 is 



C^i-{i,2}<t^{i,2}^{i,2}, (32a) 

C/i^i<t/i^{i,2}, (32b) 

with distribution 

-Pt/i^i,c/i^);i 2},c^{i,2}^{i,2} (33a) 

-^t^{l,2}^{l,2}^tri^{l,2} l'^{l,2}-,{l,2}^'^l-l|'^l^{l,2}:C/{l,2}^{l,2} ' (33b) 

while the CGRAS at decoder 2 is 

f^i^{i,2}<^^{i,2}^{i,2}, (34a) 

C^{l,2}-»2<t/{l,2}^{l,2}, (34b) 

Ui^2<Ui^[i,2}, (34c) 

;7i^2<t/{i,2}^2, (34d) 

Ui^{i,2} < C^{i,2}-2, (34e) 

with distribution 

-Fc/l^2,'^2-*2,t^l^{l,2},'^{l,2}^2,C^{l,2}-*{1.2} (35a) 

^'^{i,2}^{i,2} (35b) 

-Ptf{i,2}^2|C/{i,2}_{1.2} (35c) 

^t^2^2|C/{l,2}^2,t^{l,2}^{l,2} (35d) 

^t^i^{i,2}|c/{i,2}^2,t^{i,2}--{i,2} (35e) 

-fc/i^2|(7i^{i,2},;7{i.2}^2,c/{i,2}^{i,2}- (35f) 
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IV. CODEBOOK Generation, Encoding procedure, Input generation and Decoding procedures for a 

CGRAS 

We now outline the codebook generation, encoding, and decoding operations for a general transmission scheme associated 
with a CGRAS. The codewords are generated by drawing i.i.d. symbols according to the joint distribution associated with the 
S edges of S- They are furthermore chosen so as to appear as if generated according to the distribution associated with edges 
S and B-. Finally the channel inputs are produced as a function of the codewords known at each transmitter. At each receiver, 
the decoder looks for a set of codewords that possesses the correct conditional distribution given the received channel outputs. 
The conditions under which encoding and decoding errors vanish as the block-length goes to infinity are determined in the 
next sections. 



A. Codebook Generation 

The codebook of a CGRAS is generated according to the distribution imposed by the superposition coding edges where for 
each codeword in the base codebook a top codebook is generated. 

Given a specific coding strategy, specified by a rate- splitting matrix F and a chain graph, each message W-^^^ , the codebook 
can be generated by repeating the following step for each node in the CGRAS, 

• Consider the node f/i^j and assume that the codebook of the parent nodes has been generated and indexed by h^m G 
[1 . . . 2^'-»], that is 

C^l^m = t^l^m(^l^in), V (I,m) S.t. ?7i^j<;7l^m, 

then, for each set 

{ /i^m, V (l,m) s.t. /7i^j<i7i^m}, 

repeat the following: 

1) generate codewords, for 

(36) 

with i.i.d. symbols drawn from the distribution Pui^^\{Ui^^, t/i-,„<c/i^j}- 

2) Place each codeword C/^j in 2^^'^ bins of size 2^^'^ each. 

3) Index each codebook of size using the set { Zi^m, V (1, m) s.t. J7i_>j<f7i^m} so that 

[/i^j = C/i^j {w^, { h^m, V (1, m) s.t. Ui^i<Ui^^}) . (37) 

Since the superposition coding graph Q{V,S) is an ADG, we can apply the above step starting from the nodes that have no 
parents to all the nodes in the graph and all the way up to the leaves of graph. 
The overall distribution of the codebook {C^i'^j, V(i, j)} is therefore 

-Pcodebook = -Pq H -Pi7i^j|{J/,^„: i7i^™<i7i^„},Q' (38) 

among the RV C/i^j in the CGRAS. 

The codewords are generated in such fashion that if we consider codewords with the same set of indexes 

{ Zi^m, V (l,m) s.t. f/i^j<C/i^m}, 

are generated according to the same conditional distribution with respect to the parent nodes. 

The transmitted codeword depends therefore on the encoded message Wi^j as well as on the previously encoded 
codewords Ui^m upon which the codeword is superposed. This dependency is expressed by the conditional distribution of 
Z/j^j with respect to the parent codewords. This corresponds to the fact in superposition coding the codeword of the top 
message is chosen from a codebook that depends on the base codeword. When multiple superposition steps are performed, 
that is, multiple codewords are stacked one on top of the other, the top most codeword is generated according to a codebook 
that depends on all the codewords upon which it superposed. Li^j is the rate of each of the codebooks generated for each 
parent codeword and is in the following referred to as codebook rate. The bin index wi^j in determined by the message to 
be transmitted, we therefore refer to this index as the message index. The rate of the message index is referred to as 
message rate. The bin index hi^j in determined by the binning procedure, we refer to this index as the binning index. The rate 
of the binning index is referred to as binning rate. 
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B. Encoding procedure 

In the encoding procedure a specific codeword [//^j /i^m, V (1, m) s.t. C/i^j~C/i^m}^ is chosen for trans- 

mission. The set { h^m, V (1, m) s.t. Ui^j<U\^in} is determined by the parent codewords to be transmitted while the bin 
index Wj^j is chosen by the outcome of Wi^j . The binning index bi^j is finally chosen so that the codewords appear to have 
been generated with i.i.d. symbols drawn from the distribution 

^'encoding = ^C/i^j |{t/l^m, Ui^j<Ui^^ or (7i_j <: (7i_„},Q ' ^^^^ 

(iJ) 

instead of the codebook distribution dJST l. If a codeword cannot be determined, then it picks a value of at random. We 
may find a jointly typical codeword if the number of bins is sufficiently large, that is, if Ri^j is sufficiently large. 



C. Input generation 

Finally, node fc's encoder produces the channel input Xj^ as a deterministic function of its codebook(s), i.e. 

= Xf ({C/^j, V(iJ) s.t. fcGi}). (40) 

Restricting the class of encoding functions to deterministic functions instead of random functions can be done without loss 
of generality. This can be proved using the approach in [40] . This result is stated formally in the following theorem. 

Theorem IV.l. Restricting the encoding function in ( I40l l to be a deterministic function, instead of a random function, is done 
without loss of generality. 

Proof: The complete proof is provided in App. |C] The main intuition in the proof is that if a random encoding function 
performs well then there must be a deterministic encoding function that performs better than a random encoding procedure. 
This argument is valid regardless of the specific encoder structure. ■ 



D. Decoding procedure 

To decode the transmitted messages, each receiver z looks for a set of bin indices wj^j and bi^j for z G j, such that the set 
|Yj,^, |c^i'^j : z £ j|| looks as if generated i.i.d. according to the distribution 

Pv^AU-.^y- ^ej}' (41) 

An error is committed if any of the receivers decodes (at least) a bin index incorrectly. If the decoder cannot find such codeword 
or it finds more than one, it picks one tuple at random. 

We can find a codeword with the desired joint typicality condition; this codeword is unique if the number of transmitted 
codewords is small (if the transmission rate Li^j is sufficiently small). 

Note that the decoders, although interested only in the decoding of the messages VF/^j, are decoding all the indexes foj^j that 
index the codeword U(^y The decoding of the binning index fej^j reduces the probability of error as it requires the decoded 
messages to be distributed according to the joint imposed by the encoding procedure. 

Given the clear correspondence between the codeword [/i^j in the CGRAS and the codeword C/^j in the transmission 
strategy, in the following we often refer to one or the other quite freely. 

Also, with a slight abuse of notation, we adopt the following shorthand notation to refer to a specific codeword in the 
codebook 

fei^j) = C/i^j (wUj, ^i^j, {'v^t, V C/i^j<C/v^t}) , (42) 

We identify the codewords only with the message w[^^ and the binning index bi^j with the implicit understanding that the 
remaining bin indexes are determined by the parent codewords upon which U(^-^ is superposed. 

V. Derivation of the Rate Bounds 

In this section we derive the achievable rate region of the transmission strategy obtained from a CGRAS as described in 
Sec. |IV] The proof relies on the properties of typical sequences, that is, sequences of i.i.d. draws with lengths approaching 
infinity. For such sequences the empirical distribution as — > cxo approaches the true generating one and many properties 
follow from this convergence. In particular it is possible to bound the probability of encountering sequences that appear to be 
generated according to a different distribution than the true generating distribution. Two such bounds are: 

• Covering lemma / Mutual covering lemma |T|, |4T|, |4| bounds the minimum number of i.i.d. sequences C^i'^j, U(^^ 
that one needs to generate in order to find two that appear as if generated according to a joint distribution Rff, . jj 
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Theorem V.l. Covering Lemma f^i page 70] Consider the set of RVs [U, X, X) and the joint distribution P, 



U,X,X' 

together with the marginal distributions Pjj.x ond conditional distribution Px\u- Moreover let {U^ ,X^) be an i.i.d. 
sequence from the typical set 7f (Pfj x) ond take i.i.d. sequences X^ from the distribution P~,.- indexed as 



X (m), m G {1 . . . 2" }. Then there exists (5(e) — > e -> such that 



P 

as N ^ oo if 



3m e{1...2"«}, s.t. (;7^,x^,x^(™))eTf(P^_^,,, 



R > I{X;X\U) +S{e) 



The mutual covering lemma extends the covering lemma by considering the case where both X and X are random 
sequences. 

Theorem V.2. Mutual Covering Lemma fJ, page 221] Consider the set of RVs {U,X,X) and the joint distribution 
Pjj X X' together with the conditional distributions Px\u <^^d Px\u- Moreover let be an i.i.d. sequence from the 
typical set 7^ (Pu). Take 2^^^ i.i.d. sequences X^ from the distribution Px\u indexed as X^{mi), nii G {1 . . .2"''^^} 

and and take 2^^^ i.i.d. sequences X^ from the distribution P^,., indexed as X^ (1712), m2 G {1 . . . 2"^^ }. Then there 

X I u 

exists (5(e) — J> e -> such that 



P 



3 (mi, ms) G {1 . . . 2"«i} x {1 . . . 2"«^}, s.t. (C/^, X^ (m,), X^ (m^)) £ Tf (P^,^ o) 



as N ^ 00 if 

i?i +i?2 > I{X;X\U) + S{e). 

Detailed proofs of both the covering lemma and the mutual covering lemma are provided in [3], H. 

• Packing lemma L2J, 1321 . bounds the maximum number of independent sequences U(^j, U{^^ that one can generate 
so that no two of them appear as if generated according to a joint distribution . jj^ 

Theorem V.3. Packing Lemma |4, page 53] Consider the set of RVs {U,X,Y) and the joint distribution Pu,x,y, 
together with the marginal distributions Pjj and Px\u- Moreover let Yn be an arbitrarily distributed sequence and 
an i.i.d. sequence from the typical set 7^ {Pu)p. Take 2^^ i.i.d. sequences X^ from the distribution Px\u indexed as 
X^ {m), m G {1 . . . 2"^}. Then there exists (5(e) — > tjs e -> such that 

P[3m G{1...2"«}, s.t. (C/^,X^(a),r) GTf(P[;,x.y)] -0, 

as N --^ 00 if 

R<I{X-Y\U)~5{e). 

Detailed proofs of the packing lemma are provided in O, ||4l. 
Together with bounds on the probability of an error event, these lemmas can be used to determine bounds on the cardinality 
of the codebook and, therefore, the transmission rates. 

Both bounds are based on standard properties of jointly typical sequences [42] and are common tools used to derive achievable 
rate regions for multi-terminal networks. Our contribution lies in the generalization of the error analysis corresponding to our 
general transmission scheme for a general transmission network. 

Without loss of generality, the error analysis may be bounded as 

P [error] < 

P [encoding NOT successful] 

+P [decoding NOT successful | encoding IS successful] . 

In the following sections we provide bounds on the rates Li^j and Pi^j such that the probability of encoding error and the 
probability of decoding error given that encoding was successful, vanishes as — ^ 00. The bound on the binning rates Pi^j 
are obtained using the covering lemma of Th. IV. II and the mutual covering lemma Th. IV.2I and are of the form Pi^j > ... as 
they bound the number of excess codewords that must be generated in order for the encoding procedure in Sec. IIV-BI to be 
successful. 

The bounds on Pj^j are instead obtained using the packing lemma of Th. IV.3I and are of the form Pj^j < ... as they bound 
the maximum number of codewords that can be generated in order for the decoding procedure in Sec. |IV-D| to be successful. 

We bound both the encoding and the decoding error probability using two bounds on these probabilities. For the encoding 
probability we have 

• Error of events bound: this is a simple bound on the error probability and the results on the binning rate are not always 
tight. On the other hand the analysis of the error is conceptually simple and the bound on the binning rates have intuitive 
expressions. 
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• Markov's lemma: this is a more refined bound with respect to the previous case, but the error analysis is more laborious 
and the expressions of the bounds more involved. 

For the decoding error probability we have 

• Sequential decoding: this is a simple bound that assumes that each codeword must be correctly decoded before the 
decoding proceeds to the connected codewords in the CGRAS. The rate bounds are intuitive but not always tight. 

• Joint decoding: instead of decoding each codeword singularly, the codewords are here all simultaneously decoded. The 
resulting bounds are usually tighter than the previous case and the error analysis more involved. 

These four bounding techniques have been widely used in literature to derive achievable regions for many classic channels 
such as the multi-access channel, the broadcast channel, the interference channel, and the cognitive interference channel. 
It is commonly believed that using Markov's lemma and joint decoding yields the tightest bounds, or alternatively, the 
largest achievable regions. Despite this common belief, it is not always clear when these two bounding techniques provide 
an improvement over simpler derivations. The way in which the bounding techniques compare is relevant not only from a 
purely theoretical standpoint, but is also relevant in the design of practical communication systems. The bounds using Markov's 
lemma and joint decoding suggests that encoding and decoding should be done in a simultaneous fashion while the bounds 
using the error of events bound and sequential decoding point toward a sequential implementation. With our analysis we are 
able to answer this longstanding question for a general achievable scheme by deriving the rate bounds obtained with each of 
these four approaches. 

A. Encoding Errors 

For the probability of encoding error to vanish as the block-length increases it is necessary to choose a large enough binning 
rate so that it is possible to jointly find a set of bin index for which the codeword appears generated according to 
the conditional distribution in ( l39l ) although generated according to the conditional distribution in ( |38] |. This bound is obtained 
using the covering lemma which bounds the minimum number of typical sequences that must be generated in order to be 
able to find one that "fakes" or "deceives" a probability distribution. Intuitively an encoding error occurs whenever it is not 
possible to find a bin index bj^j so this deceiving codeword can be found. The encoding error probability then depends on all 
of the possible combinations in which an encoding error may be committed, that is on all the possible ways of failing to find 
the appropriate bin index of any given codeword or, equivalently, jointly typical codewords. The probability of any specific 
encoding error event depends on two factors: the number of excess codewords generated and the joint distribution that we 
are trying to fake. The number of excess codewords in the bin corresponding to the index &i^j is determined by the binning 
rate Ri^j and choosing large enough grants that a deceiving codeword exists. The difference between the codebook 

distribution and the encoding distribution determines how hard it is to find the appropriate codeword; this distance is intuitively 
related to the KL divergence between the distribution imposed by the encoding and the actual distribution in the codebook. 

This intuitive interpretation of the error probability corresponds to the derivation of bound on the binning rates obtained 
using the union of events bound. The bound obtained using Markov's lemma has a more complex structure and does not lend 
itself to a simple interpretation. 

1) Some more notation: We introduce here some additional notation that we use to compactly describe results in the 
following section. 

Since an encoding error is committed whenever it is not possible to find the desired bin index 6i^j for any of the codewords 
C/j'^j, the overall probability of error is obtained considering all the possible permutations in which an encoding error can be 
committed over the set of all codewords. It is actually necessary to consider only the codewords with a non-zero binning rate, 
thus it is convenient to define the set Sb as: 

Sb = {(i, j), Ui^i < Ui^m for some (1, m)} , (43) 

Sb is the set of all the indexes whose codeword possess a bin index. An error event is defined by any subset S C Sb as the 
error event where the codewords C/j^j for (i, j) e S are incorrectly encoded while the codewords in the complement of the set 
S in Sb are correctly encoded. 

Intuitively we expect the condition for a successful encoding to depend on the distance between the codewords' distribution 
at generation and after encoding: 

Cdcbool = £^[l0g ^encoding] -^[logPcodcbook] = ^ I (Ui^y, {Ul^m, C/i^j < C/l^m} | {f/l^m , C/i^ < C/l^m} | Q) ■ (44) 

(iJ)eSB 

In the following section the term I™^cbook '^^'^ related terms provide some insight on the nature of the conditions under which 
encoding is successful. 
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2) Encoding errors analysis using the union of events bound and the covering lemma: We begin by deriving the constraint 
on the binning rates using the union of events bound and the covering lemma of Th. IV. 11 

Theorem V.4. Encoding errors analysis using the union of events bound and the covering lemma: For any CGRAS, the 
encoding procedure described in Sec. \IV-B\ is successful with high probability as N ^ oo if , far any subset S C Sb such that 

(ij) e S ^ (l,m) e S, V (l,m) s.t. [/i^^<f/i^j, (45) 

for Sb in ( 1431 ), the following holds: 

J2 >lg '^-, (46) 

(iJ) 6 S — roots 

and where Ig '^' is defined as 

Ig-L- = Es-rootB [log ^oncodingls] " Es-roote [log ^'codcbookl s] (47a) 

J2 I{Ui^y,{Ui^m S.t. (l,m)eS}|{t/i^™ S.t. C/i^j<C/,^„,C/i^^<C/i^j,(l,m)eS},Q), 

(ij) G S— roots 

(47b) 

fC.L. standing for "Covering Lemma" ) where Es-roots indicates the expectation is taken over C/i^j in S — rootB that is 
defined as 

S - rootB {Ui^i s.t. ^ (1, m) e S, C/i^j<C/i^m}, (48) 

and where 

p _ _ f encoding .„ , 

-^oncodinglS ^ TT - P ^ ' V^^''-) 



p _ _ ^codcbook , . „, , 

^r.odfihooklS ~ tt n ■ (^yu) 

l(iJ)€ 



codcbookl S TT p 



Proof 

The complete proof can be found in App. |D1 we provide only the first steps of the proof and an intuitive interpretation of 
the remaining passages. 

The probability of encoding error can be bounded using the union of events bound as the sum over all the possible subsets 
of Sb as defined in ( l43T i: 

P [encoding NOT successful] < ^ P [i;s] , (50) 

ScSs 

where the event Es is defined as 

Es= C/i^j NOT correctly encoded, (iJ) e S | f/j^j correctly encoded, (i, j) £ S. 

The bound in dSOl l can be tightened by noticing that certain error events need not be considered in the summation of dSOb . 
The codeword U^^^ is taken from a codebook determined by all the codewords such that Ui^j<U\^irf If the encoding 

of any of the fails, the codebook from which is picked does not have the right conditional distribution given U^^. 

For this reason it's impossible to find a codeword that possesses the desired joint distribution. We can formally state this 
reasoning as follows: let Es' be the error event in which C/^j is correctly encoded, i.e. (i, j) G S, while in incorrectly 

encoded for some C/i^j~?7i^m, i e. (1, m) G S. Given the encoding procedure of the CGRAS 

P[Es']<P[Es,u(i,i)]. (51) 

the codebook for depends on the codeword as expressed in ([37]). 

If the encoding succeeds with high probability, then 

lim P [i?s'u(i,j)] ^ 0, 

A* — >oo 

which implies that 

lim PlEs'] 0, 

AT-i-oo 

as well. 
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Condition ( |45T l specifies the set of S for which the error probability P[Es] cannot be bounded as in (|5TT i and we thus 
conclude that 

lim V P [Es] ^ ^ lim V P [Es] ^ 0. (52) 

AT— >.oo ^ — ' JV— !-oo ^ — ' 

SCSb ScSb, [45] 



In the following we will derive the condition under which the RHS of (152b occurs. 

The same consideration that leads (|52] | is also key in bounding the probability P[Es]'. since an error in encoding the base 
codeword induces an error in the top codewords, the probability of P[i?s] corresponds to the probability of committing an 
error in encoding the base codewords of the set S, that is, the codewords in the set S rootB in (|48] |. When the codewords 
in the set S — rootB are all incorrectly encoded, all the codewords in the set S are also incorrectly decoded as superposed to 
incorrectly encoded codewords. 

For each set S such that ( |45] l holds, the bound on the binning rates depends on the difference in the distribution of the 
codewords in S at generation and the distribution after encoding, given all the correctly encoded codewords. This difference is 
intuitively measured by the mutual information term ( |47] i which quantifies the distance between the distribution ( |49bt and (149 at , 
which are the distribution of the incorrectly encoded codewords at generation and after the encoding operation respectively. 

In App. |D]we prove that under the conditions in i45[ the encoding of the codewords of S — roote is successful with high 
probability as N ^ 00. 

■ 

Remark V.5. The term Is'^' in ( l47b quantifies the distance between the distribution at generation and the distribution after 
encoding for the set S — roote. This term has a very similar expression to the term I™dcbook < l44b when the summation is 
restricted to the terms in the set S — rootB. An important difference is for the terms that have joint binning, in which case an 
extra term appears in the conditioning. 

Mutual binning indeed makes the codeword appear as if it was generated according to a distribution different from the 
codebook distribution, despite of failed encoding. Consider for instance the CGRAS where 

C^{i.2}^i < C^{i,2}^2, (53a) 

C/{1,2}^2<C/{1,2}^1, (53b) 

and the set where the encoding of ?7{i,2}-^i f^ls but the encoding of C/{i,2}^2 is successful, that is 

({l,2},l)eS, (54a) 
({l,2},2)eS. (54b) 

(54c) 



Assume also that condition (l45l l holds for S. In this case the joint distribution between the codewords C/{i,2}^i and ?7{i,2}-+2 
after encoding is Pu^^ 2}^i C/{i 2)^2 if '^^e encoding of one codeword fails. 
3) Bounds on the binning rates using the Th. \V.4\ for CIFC: 

To illustrate the result in Th. IV.4I we consider the CGRAS in (|30g|l, also depicted in Fig. |6] 



The scheme in (30gi has three codewords performing binning: f/i^{i.2}, Ui^i and t/1^2 and thus 

Sb = {(1,{1,2}),(1,1),(1,2)}. 

All the possible subsets of Sb are provided in Tab. II VI in this table the column indexed by (i, j) G Sb indicates whether 
the corresponding bin index has been successfully encoded, using / , or not , using x or H. The distribution of the codewords 
after the failed encoding procedure, i.e. ^cncoding|S' i^ indicated in the last column, where Pprimary is defined as 

Pprimary — ^;7{ 1.2} {1,2} 1.2} 1,2} ^2 ■ 

The distribution ^'codcbook|s is obtained by first obtaining the conditional distribution of Pcodcbook in ( 1381 ) given the correctly 
encoded codewords. The codewords in the set S — rootB defined by ( |48] | are indicated with the symbol Kl. 
The rate bounds obtained applying Th. IV. II to this CGRAS are: 

-Rl^{l,2} > (^^1^(1,2}; t^{l,2}^2|C^{l,2}^{l,2}j Q) (55a) 
Rl^l + i?1^2 > I {Ul^i; Ui^2, C^{1,2}^2|C^{1,2}^2, C^1^{1,2}, ^^{1,2}^{1,2}, Q) (55b) 

Ri->2 > (55c) 

> I (C^l^i; C^{l,2}-i-2|C^{l,2}^2- t^l^{l,2}, t^{l,2}^{l,2}, Q) , (55d) 
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TABLE IV 

Encoding error event table using Th. |V.4| for CIFC in j30g( 





^1^{1,2} 




bi^2 


p _ 

cncodingi S 


El 


X 


/ 


/ 


(451 does not hold 


E2 




/ 

V 




(45 1 does not hold 


E3 


X 


X 


/ 


(451 does not hold 


E4 


m 


X 


X 


P P 

P"m'»'^y Cl^{l,2} |f {1,2}^{1,2} 
-^fl^ll^i^^^l 2} .'^{1,2}^{1,2} 
-^'^1^2|tfl^{l,2} .'^{1,2}->2.C{1,2}->{1,2} 




/ 






P ■ P 

-rprimary-rj7j_^^ l!^{l,2}->{l,2}.'^{l,2}->2 
-^['l-n|Cl^{l,2} .'^{1,2}^{1,2} 
-'^!^l->2|Cl_>f l,2}''^f l,2}^2>t^f l,2>^f 1,2} 


E6 


/ 


/ 




p . 

^ primary 

-^['l-H,Ui_>2C/l_>{l,2) It'f l,2}^.f l,2}>'^{l,2>-+2 


E7 


/ 


Kl 


/ 


-PprimaryPi7i^^l_2}|C^{l,2}^{l,2}.'^{l,2}~>2 

-^!^l-nlC'l^{l,2} .'^{1,2}^{1,2} 
p 

-^['l-f2|Cl^l.'^l^(l,21'C'{l,2>^2.'^{l,2|^{l,2| 


E8 


/ 


/ 


/ 


encoding is successful 



B. Encoding errors analysis using Markov inequality and the mutual covering lemma 

By employing Markov lemma, one can obtain a more refined error analysis than the one offered by the union of event 
bounds. The bound on the binning rates under which encoding is successful are generally better than the one obtained using 
Th. IV. II but these bounds are usually less intuitive. This analysis requires mutual covering lemma of Th. IV.2I 

Theorem V.6. Encoding errors analysis using Markov inequality and the mutual covering lemma: For any CGRAS, the 
encoding procedure described in Sec. \IV-B\ is successful with high probability as N ^ oo if, for any subset S C Sb such that 



(ij) e S (l,m) e S, V (l,m) s.t. C/i^^<C/i^j, (56) 
for Sb defined in (I43l l, the following holds: 

E-Jy ^ Tcncoding tM.C.L. /c-tx 
-Ki^ ^ Icodcbook ~ ■'■S ; y^') 

(iJ) e s 

where 

I^-C-L- = Es [log P,„,„di„g|s] - Es [log Peodebookis] (58a) 

= J2 HUi^y, {Ui^m, Ui^i<Ui^m, (1, m) e S}|{C/i^^, f/i-,j<f/i_™, Ui^n, < Ui-,j, (1, m) e S}, Q), (58b) 
(iJ) e s 

(M.C.L. standing for "Mutual Covering Lemma") where Es indicates the expectation is taken over Ui^j in S and where 
^cncodingls ^codobook|s "'"^ defined as in ^ and I™dcbool '« S 
Proof: 

The complete proof can be found in App. |E1 we provide here the first steps of the proof and an intuitive interpretation of 
the remaining passages. 

While in Th. IV. 4 1 we utilize the union of events bound to analyze the encoding error probability, here we use the Markov 
inequality. By using this bounding technique we are unable to readily link a bound on the binning rate Ri^j with a specific 
error event in Th. IV.4I Instead, the binning rate bounds relate to the calculation of variance and mean of the encoding error 
event. 

We start the proof by applying the Markov inequality to the encoding error probability as follows: 

P[encoding NOT successful] = P bi^j, s.t. {U-^^{w^,bi^i), (iJ) e Sb, 6i^j e bi^j} e Tf (p™™di„g-)j ^59^^ 



n (iJ) e Sb, fei.j e bi^j} e (P<=-oding) 



(59b) 
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TABLE V 

Encoding error events table using Th. |V.6| for CIFC in pOg) 





''l^{l,2} 


61^1 


61-^2 


JM.U.L. foj. S 


El 


X 


/ 


/ 


(561 does not hold 


E2 


X 


/ 


X 


(561 does not hold 


E3 


X 


X 


/ 


(561 does not hold 


E4 


X 


X 


X 


^{'^1^{1,2});C/{1,2}^2|C/{1_2}-.{1,2} + 


I{Ul^i; Ul^2, C^f l,2}^2|f^l^{l,2}: f^{l,2}^fl,2}) 


E5 


/ 


X 


X 


I{Ul^i;Ui^2, t^{l,2}^2l'^l^{l,2}. f^{l,2}^{l,2}) 


E6 


/ 


X 


/ 


I{Ul-,i;Ul^2, t'{l,2}^2l'^l^{l,2|) C^{l,2>^2i '^{l,2}-.f 1,2}) 


E7 


/ 


/ 


X 


/(C/1^2; t^l^l|f^l^fl,2>>!^{l,2}^2.f^fl,2>^{l,2}) 


E8 


/ 


/ 


/ 






where 



bi^j = (ij) e Sb} V&i^jG 



1...2 



Kb;^, = 1{£}, 

tN 



NR, 



£ = {U,'^^{wi^j,h^i), (iJ) e Sb, e bi^j} e Tf (Pencoding) . 



(60a) 
(60b) 

(60c) 
(60d) 



The probability of encoding error is equivalent to the probability that no choice of bin indexes bi^j produces the desired 
typicality. We then associate the indicator function Ki^j to the event that a sequence of bin indexes bi^j produces a successful 
encoding and the indicator function K to the event that encoding is successful. By applying the Markov lemma to the indicator 
function K, we can obtain bounds on the binning rates by estimating the mean and variance of K. 

The mean E[K] is 

and can be interpreted to the fact that, on the first approximation, the encoding event is more likely as the number of excess 
codewords in the codebook increases, but less likely as the distance between the encoding and the decoding probability grows. 

The variance Var[K] does not have a compact expression but it relates to the correlation between the events that the two 
sequences bi_»j and bi^j produce a successful encoding. This correlation depends on how the typicality condition £ in (I60db 
varies among the two sequences. 

In calculating Var[K] we encounter the same condition as (l45T l and we therefore link the notation of Th. IV4l and lV.6l through 
this condition. Unfortunately it is not easy to relate the binning bounds obtained with these two theorems as straightforwardly 
as one may wish. 

In App. |E] explicitly derive E[K] and Var[K] and derive the conditions under which "^^il^ - j goes to zero as TV — > cxi. 



Condition (l56T l is equivalent to condition ( |45] | and is restated here only for clarity. 
Remark V.7. As in Rem. |V.5l we point out to the reader the role of mutual binning in the conditioning of the term l|i C.l. 
(|58b| i. This term is derived in a similar fashion than Ig '^ in Wf\ and the same reasoning in Rem. |V.5| holds here as well. 

1 ) Bounds on the binning rates using the Th. \V.6\ for CIFC: 

As for Sec. IV-A3I we illustrate the result in Th. IV.6I by considering the CGRAS in (30gi, also depicted in Fig. |6] The 
achievable scheme and some basic notations are introduced in Sec. I V-B 1 1 

Icodcboox = -^(t^l^{l,2}; t^{l,2}^2|C^{l,2}^{l,2}, Q) + -^(C^l^i; Ui^2, C^{l,2}^2|t^l-»{1,2}, C^{1,2}^2 , C^{1,2}^{ 1,2} , Q) , (61) 

while the terms I|I C L- derived in Tab. |V] Tab. |V]is organized in a similar manner to Tab. IIVI the column indexed by 
(i,j) e Sb indicates whether the corresponding index (i,j) is in S, using /, or in S, using x. The term c for each 
possible S is indicated in the last column. 

Using Tab. [V]we come to the following bounds on the binning rates: 

^1^(1,2} > ^(t^l^{l,2};C^{l,2}^2|t^{l,2}^{l,2},Q) (62a) 
i?l-{l,2} +i?1^2 > /(t/l^{l,2});C/{l,2}^2|C/{l,2}^{l,2},Q) (62b) 
-Rl^{l,2} + -Rl^l > I{Ui^{i,2}); C^{l,2}^2|t^{l,2}^{l,2}, Q) 

+ /(C/i^i; (7i^2|(7{i.2}^2, U,^{ia}, C/{i,2}^{i,2}, Q) (62c) 

^1^(1,2} + Rl^2 + Rl^l > I{Ui^{i,2}; C^{l,2}^2|t^{l,2}^{l,2}, Q) 

+ liUi^i; Ui^2, C^{1,2}^2|C^1^{1,2}, t^{l,2}^2, C^{1,2}-{1,2}, Q)- (62d) 
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Note that Tab. |IV] is very similar to Tab. |V) given the similarity in the proof of the two theorems, the codebook bounds are 
derived in a very similar manner In the following we will use only one table to represent both cases by simply adding the 
last column of one graph to the other. 



C. Decoding Errors 

For the decoding error probability to vanish as the block length increases, it is necessary to choose a small enough codebook 
rate Li^j so that the codewords C//^j are sufficiently "spaced apart" in the typical set to allow successful decoding. The 
bounds on the codebook rates are obtained using the packing lemma which bounds the maximum number of codewords that 
can be employed at transmission and still allow the decoder to recover the transmitted codeword. Decoding relies on the 
conditional typicality of the channel output given the transmitted codeword and on the typicality relationship imposed on 
the codewords by the encoding procedure. More specifically, a decoding error is committed when the decoder finds a set of 
wrong codewords U^^{'w?^j,b^^^) for {^^^^,bf^^) ^ (^f-+j' ^f-^j)' '^^at is jointly typical with the channel output and jointly 
typical with the correctly decoded codewords according to the distribution at encoding. An decoding error happens when the 
random effects of the channel alter the channel output in such a way that 1^;,^, {U^j, V (i, j) s.t. z G j}| appears to be 
generated according to the distribution Pencoding, despite an error in selecting the bin or message indexes of some codewords. 
Since codewords are generated in an i.i.d. fashion, the incorrect codewords are independent from the channel output and 
thus the channel effectively introduces a dependency between independent sequences. In this perspective the channel acts a 
binning encoder that randomly introduces correlation among conditionally independent codewords Ul^j{wf^j,b^^^) and V/^ . 
The incorrectly decoded codeword (/(^^{wi^j, does not possess typicality properties with the correctly decoded codewords 
as produced at the encoding, when the bin indexes bi^j are picked. When an error is committed in decoding a bin index, 
then the resulting set of codewords loses the typicality property obtained through encoding. With these considerations we see 
that the probability of an encoding error intuitively relates to the KL divergence between the joint distribution of the channel 
output Yz and the codewords as they appear after encoding and the distribution of the channel output times the distribution of 
the codewords they appear in the codebook. The analysis of the decoding errors is similar to the encoding counterpart in that 
we need to consider all the possible permutations in which a decoder can fail in determining the message index Wi^j and the 
bin index In general the error permutation of decoding errors provide an intuitive interpretation of the bounds in terms 
of conditional mutual information. 

1) Some more notation: We introduce here some additional notation that we use to compactly describe results in the 
following section. 

The probability of error at each decoder is linked to the specific codewords that the receiver is attempting to decode; for 
this reason it is convenient to define the set as: 

S^ = {(iJ) s.t. zGj}, (63) 

that is, is the set of indexes whose codeword is decoded at receiver z. An error event at decoder z is defined by any 
subset S C Sf) as the error event where the codewords Ui^j for (i, j) G S are incorrectly decoded while the codewords in the 
complement S in Sfj are correctly decoded. 



Intuitively we expect the condition for successful decoding to depend on the distance between the joint distribution of the 
codewords and the channel output at the encoding and the distribution after an incorrect decoding, that is: 

IcZd!:;! = i?[logPy.,c„codi„g--] - iJilogi^y./'codcbook] 



where 



iV,, encoding^ — | {C/i^j eS£,},Q^'oncoding (65a) 

^encoding = ^ ' -Poncoding (65b) 

^codebook ~ ^ ^ Pcodebooki (65c) 

that is, -Pelicoding ™d -Pcodebook encoding and the codebook generation distribution, as provided in ( [39] l and 

respectively, for the set of codewords decoded at decoder z, that is for the set {Ui^j, (i, j) £ Sfj}. 
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2) Decoding error analysis using sequential decoding and the packing lemma: We begin by deriving the codebook rate 
bounds for the sequential decoding case. This is a simple way of analyzing where one uses the fact that a codeword can be 
correctly decoded only if all the codewords on which it depends have also been correctly encoded. The packing lemma of Th. 
IV.3I provides the theoretical tool to bound the error probability. 

Theorem V.8. Decoding error analysis using sequential decoding and the packing lemma: For any CGRAS, the decoding 
procedure described in Sec. \IV-D\ is successful with high probability as N —i- oo if for any receiver z and for any subset 
S C such that 

(ij) e S ^ (l,m) e S, V (l,m) s.t. [/i^^<C/i_j, (66) 
for Sf) defined in (|63] ). the following holds: 

(iJ)eS"'-rootD 

where 

Is ^'^ = lEy. -rootD [log Py^^ oncodingis] ^ -rooto docodingls] 

- /(n; {[/i^j, (i, j) e - rootD}|{{/i^m, (1, m) e S}, Q) 
+ ^ /(C/i^j;{;7,^m, C/i^j^C/i^m, (l,m) e S^-rootD}|{C/,^m, C/i^j<C/i^m e S},Q), (68) 

(i.j) 6 S^— rooto 

{where S.D stands for sequential decoding) and where Ey g_j.QQj indicates the expectation is taken over the channel output 
Yz and J7i^j G S — roote that is defined as 

- rootD = {C/i^ s.t. ^ (1, m) G S, C/i^j<C/i^™, t/,^™ < C/i^j}. (69) 

and where 

pz 

ry encoding ,^ 

encodingIS " | {C/i^j eS£, } ' ff ^ (/Ua) 

ll(iJ)GS -^;7i^j|{;7i^„, (7i^j<C/i^„ or Ci^j'^C/i^m} 
pz 

p p codcbook t^OV\\ 

^Y;,, dencodinglS " ^YJWi^iGS} ' TT P ' K'^^) 



Proof: 

The complete proof can be found in App. |F1 we provide here the first steps of the proof and an intuitive interpretation of 
the remaining passages. 

Without loss of generality assume that the message set = 1, V (i,j)} is to be transmitted and that the encoding 

procedure and that the set of bin indexes {bi^j = 1, V (i, j) G S|,} is chosen at the encoder. It is sufficient to prove Th. 
IV. 8 1 for a single decoder z, since the decoding operation is analogous for all the decoders. A decoding error is committed at 
decoder z whenever either the message or the bin index of any given codewords fails, that is, whenever 

(w-UyM^j) ^ (W^i->j:^i^j) / (l.^i^j)- 

Note that, in general, a decoder is not interested in decoding the binning index but only on the message index w\^y 
Despite this, the correct decoding of the binning indexes is necessary in two situations: 

• when another codeword depends on thorough superposition coding: when the codeword U^_^^ is superposed to JZ/^j, 
then its codebook depends on the bin index h\^y 

• when another codeword depends on thorough binning: when the codeword U\_^^ is binned gainst then the 
choice of the bin index 6i^m depends on the bin index as well. 

For the cases where it is not necessary to correctly decode the binning index fei^j, one can sometimes show that not decoding 
&i^j does not enlarge the achievable region |23|. Codewords indexed with different fei^j representing the same message w^^j 
have no particular relationship among each other, therefore one can intuitively expect that decoding the correct luj^j can be 
done only by identifying the specific transmitted codeword J7i^j(wJ^j, Therefore correctly identify the binning index 

6i^j does not impose additional constraint on the codebook rate. 

An error is committed when there exists a set |(-5;i_^j, 6i^j), G S, (wi^j, &i^j) 7^ (l5^i^j)| such that 

(n, {[/i^C^ii^j, fei^j), (i, j) G S} , (i, j) G S}) G Tf (r,, {[/i^j, (iJ) G S^}) , (71) 
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Similarly to the proof of Th. IV.4I the probability of a decoding error can be bounded using the union of events bound as 
the sum over all the possible subsets of Sd as defined in (|63] i: 

P [decoding NOT successful at Dec. z|encoding NOT successful] < ^ P [Ds] , (72) 
where the event Es is defined as 

Ds = NOT correctly decoded, (i,j) G S | U-^j correctly decoded, (i,j) £ S, U-^j correctly encoded, (i,j) g Sf,. 

The bound in (f72b can be tightened by noticing that certain error events need not be considered in the summation of ( |72] |. 

The decoder looks for the codeword in a codebook that is determined by all the codewords such that Ui^yKUi^m- 

If the decoding of any of the fails, the codebook in which the decoder looks for does not contain the actual 

transmitted codeword. For this reason it's impossible to correctly decode with high probability. Similarly, the decoder 
looks for a codeword that has the joint typicality properties imposed by the encoding procedure. The choice of the binning 
index is determined by all the codewords such that C/i^j < f7i-,m- If the encoding of any of the fails, the 

transmitted codeword does not have the right conditional distribution, given U^^. For this reason it's impossible to correctly 
decode the codeword U^^y 

We can formally state the above reasoning as follows: let Ds' be the error event in which is correctly decoded, 
i.e. (i,j) G S, while is incorrectly decoded for some Ui^j<Ui^m, i e. (1, m) e S. Given the encoding and decoding 

procedure of the CGRAS 

P [Ds'] < P [^s'u(i,j)] , (73) 

since the codebook for U(^^ depends on the codeword as expressed in ( [37] ). 

If the decoding succeeds with high probability, then 

^linr^P[i?sMi,J)]^0' 

which implies that 

lim P \Ds'] 0, 

as well. 

Condition ( |66] | specifies the set of S for which the error probability P[Es\ cannot be bounded as in (l73T l and we thus 
conclude that 

lim V P \Ds\ lim V P \Ds] -> 0. (74) 

In the following we will derive the condition under which the RHS of (|74] | occurs. 

As for Th. IV.4I the same consideration that leads ( |66] l is also key in bounding the probability P[_Ds]- 
A sufficient condition for the event Z?s to occur is to have an error in decoding the base codewords of the set S, that is, 
the codewords in the set S — rooto in ( l69] l: since an error in decoding the base codeword induces an error in decoding all the 
related codewords, the probability of -P[-Ds] is upper bounded by the probability of committing an error in decoding the base 
codewords of the set S, that is, the codewords in the set S — rootD in ( |63] |. When the codewords in the set S — rooto are all 
incorrectly encoded, all the codewords in the set S are also incorrectly decoded as superposed to or binned against incorrectly 
decoded codewords. 

For each set S such that (|66] | holds, the bound on the codebook rates depends on the difference in the distribution of the 
channel output and the codewords in S at encoding and the distribution after incorrect decoding, given all the correctly encoded 
codewords. 

This difference is intuitively measured by the mutual information term ( IV.SI l which quantifies the distance between the 
distributions (I70ab and (I70bb . which are the distributions of the channel output and the codewords in S when correctly decoded 
and when incorrectly decoded, respectively. In App.|F]we prove that under the conditions in ( |66] | the decoding of the codewords 
of S — rooto is successful with high probability as — oo. 

■ 

Remark V.9. In the term Ig the mutual binning term does not appear in the conditioning as for Th. IV.4I and IV. 6 1 When 
a bin index (1, m) g S is correctly decoded, this does not change the distribution according to which the decoded codewords 
appear to possess. Let U^ty (iij) G S such that U\^m < Ui^j, then (f^/Zji t^rlm) appear to be generated according to the 
distribution at encoding only when they are both correctly decoded, i.e. when they are both in S. 
Remark V.IO. Note that, given S, — rootD is not uniquely defined. For instance consider the case 

C^i^{i.2}<t^i^i, (75a) 
C/i^i<C/i^{i,2}, (75b) 
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TABLE VI 

Decoding error event table using Th. |V.8| for CIFC in j30g) for decoder 1 in Sec. |V-C3] 





f^{l,2}-{l,2} 


'^1^{1,2} 




p — 

. dcncodingIS 


Dl 










D2 


/ 


m 




"'^yi|C{l,2>^{l,2> 


D3 


/ 


/ 




"^yi|Cl-{l,2} .f{l,2}-fl,2} 



TABLE VII 

Decoding error event table using Th. IV.SI for CIFC in )30g) for decoder 2 in Sec. IV-C31 





'^{1,2}-{1,2} 


'^{l,2}-+2 


'^1-{1,2} 


Ul-,2 


C-^2^2 


p — 

Yj: , dcncoding S 


Dl 














D2 


/ 










p 

^1 '^{l,2}-.f 1,2} 


D3 


/ 


/ 






m 


p 

5^1 '^{l,2|-{l,2}>C'{l,2}-.2 


D4 


/ 


/ 






/ 


P^l '^{1,2}-{1,2} .'^{l,2}-2.'^2-2 


D5 


/ 


/ 


/ 






p 

5^1 '^{l,2|-{l,2|>'^fl,2}-2.'^l-}1.2} 


D6 


/ 


/ 


/ 


/ 


Kl 


p 

5^1 '^{l,2}-.{l,2}.C^{l,2}-2.'^l-{l,2}.'^l-.2 


D7 


/ 


/ 


/ 




/ 


p 

^1 '^{1,2}-} 1,2} >C^{l,2}-.2.'^l-.{l,2}.'^2-.2 


D8 


/ 


/ 


/ 


/ 


/ 


successful decoding 



then, for S = {(1, 1), (1, {1, 2})}, we can define - rootD = {(1, 1)}, as well as - rootD = {(1, {1, 2})}. 
3) Bounds on the codebook rates using the Th. W.8\ for CIFC: 

As for Sec. IV-A3I we illustrate the result in Th. IV.6I by considering the CGRAS in (30gi, also depicted in Fig. |6] The 
achievable scheme and some basic notations are introduced in Sec. IV-B 1 1 

The set to address all the possible permutations of decoding errors Sj-, and are defined as 



{({1,2},{1,2}),(1,{1,2}),(1,1)} 



S^ = {({1,2},{1,2}),({1,2},2),(1,2),(2,2)}. 



The term lf^™ding for *is CGRAS at decoder 1 is 



Tdccoding 1 
encoding 



while at decoder 2 it is 



rdecoding 2 



encoding 
rz, S.D 



I(X2] '^{L2}^{L2}, t^{l,2}^27 ^^1^2, U2^2\Q) + I{Ui^{i^2}] t^2^{ L2} I t^{l,2}^{l,2} , Q) , 



(76a) 
(76b) 



(77) 



(78) 



while the terms Ig can be derived using Tab. IVII and Tab. IVIII In these tables the column indexed by hi^j, (i,j) e Sd 
indicates whether the corresponding bin index has been successfully decoded, using /, or not , using x or The elements 
in — rootD are indicated with Kl while the element is S but not in — rooto with x. The symbol ". . ." on the column of 
f/i^j indicates that among all the possible values S containing the other elements in the row, only when (i, j) e S condition 
(l66T l holds. In other words this symbol indicates that the codewords on a given row not marked with . . . determine an error 
in decoding the elements marked with ". . .". The distribution of the channel output for the specific decoding error pattern is 
indicated in the last column. 

Using Tab. |VT] we come to the following bounds on the codebook rates at decoder 1 : 



-^{1,2}^{1,2} < -^(^i; t^{l,2}^{l,2}IQ) 

-^1^{L2} < -^(^i; t^l^{L2}l^^{L2}^{L2}7 Q) 

Ll^l < /(Yi; C/l^l|C/l^{l,2}, '^{L2}^{1,2}, Q) 

Tab. IVIII is useful in deriving the codebook rates at decoder 2: 



^{L2}^{L2} < I{Y2 
L{1,2}^2 < I{Y2 
-^1^(1,2} + -^2^2 < I{Y2 

Li^{u2} < i{y2 

Ll^2 + -^2^2 ^ 1^X2 
L2^2 < I{Y2 
Ll^2 < I{Y2 



U{1,2}^{1,2}\Q) 
C^{1,2}^2|C^{1,2}^{1,2}, Q) 

C^1^{L2}, C^2^2|C^{1,2}^{1,2}) + -^(t^l^{l,2}, C^{1,2}^2 I L2}^{ L2} , Q) 
t^l^{L2}|t^{l,2}^{l,2}) + -^(t^l^{L2}, t^{L2}^2l'^{l,2}^{L2}, Q) 
C^1^2, t^2^2|?/{l,2}^{l,2}7 ^^{1,2)^27 ^^1^(1, 2} : Q) 
Ul^2, Jv'2^2|t^{L2}^{L2}, ^^{1,2}^2, ^^1^{1,2}, Q) 
Ul^2, '^2^2|t^{1.2}^{L2}, ^^{1,2)^2, ^^1-.{1,2}, C^2^2, Q) 



(79a) 
(79b) 
(79c) 

(80a) 
(80b) 
(80c) 
(80d) 
(80e) 
(80f) 
(80g) 
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TABLE VIII 

Decoding error event table using Th. IV. 1 II for CIFC in \30g) for decoder 1 in Sec. |V-C5] 





^fl,2>-{l,2} 






distribution 


Dl 


X 








D2 


/ 


X 




■Pyiic'{i.2>^{i,2> 


D3 


/ 


/ 


X 





where 



4) Decoding error analysis using joint decoding and the packing lemma: The derivation of the codebook rate bounds using 
the sequential decoding is conceptually simple and, as for Th. IV.4I it provides an intuitive understanding on the condition that 
allows successful decoding. Joint decoding is a more sophisticated bound on the decoding error events and provides better a 
bound on the codebook rates. Again, the packing lemma of Th. lV.3l provides the theoretical tool to bound the error probability. 

Theorem V.ll. Decoding error analysis using joint decoding and the packing lemma: 

For any CGRAS, the decoding procedure described in Sec. \IV-D\ is successful with high probability as N ^ oo if , for any 
receiver z and for any subset S C such that 

(ij) e S =^ (l,m) e S, V (l,m) s.t. C/i^^<C/i_j, (81) 

for defined in ( 1631 ), the following holds: 

E ^-j^Is°, (82) 

(iJ)GS 

''-^ = Ey^ , S [log Py^ , cncodingis] " , S [log Py^ ^dccoding|s] (83a) 

= /(n;{{7i^j,(iJ) e S}|{{/i^^, (l,m)eS},g) (83b) 

+ J2 HUi^jAUi^n,, Ui^j^Ui^^, ilm) eS}\{Ui^^, Ui^j<Ui^^ e S},Q), (83c) 
(iJ) e s 

(where J.D stands for joint decoding) and where Ey,. s indicates the expectation is taken over the channel output Yz and 
f/i^j G S and where Py^ c„coding|s Py,, c„coding|s ^^fi^^d as in (ES. 

Proof: 

The proof of Th. IV. Ill is very similar to the proof of Th. IV. 8 1 The difference is the bounding of the term P[£'s]- In Th. IV. 8 1 
we bound the probability of specific decoding error sequence Ds as the probability that the codewords indexed by — rootD 
are incorrectly decoded and then use the packing lemma for this set. In Th. IV. Ill instead, we apply the packing lemma to 
the elements indexed by S directly. Despite the fact that Th. IV. Ill conceptually provides a more refined bound on the error 
probabilities than Th. IV.8I we are not able to compare the bounds on the codebook rates obtained with these two methods for 
a general channel. The complete proof is provided in App. |G] ■ 

Condition (l66T l is equivalent to condition dSTl i and is restated here only for clarity. 

Remark V.12. Although the rate bounds in Th. IV.8l and Th. IV. 11 1 have similar expressions, it is not straightforward to compare 
the corresponding rate regions for the general channel model we consider. It is generally believed that the Th. IV. 1 1 1 yields a 
larger bound for the codebook rates than Th. IV. 8 1 since the former theorem is based on a better bound of the decoding error 
probability. 

5) Bounds on the codebook rates using the Th. \V.11\ for CIFC: 

As for Sec. IV-A3I we illustrate the result in Th. IV.6I bv considering the CGRAS in (30gi, also depicted in Fig. |6] The 



achievable scheme and some basic notations are introduced in Sec. I V-B 1 1 Following the notation in Sec. IV-C3I we use Tab. 
(Ivml l and dHl to calculate the terms Ig' "''^ in (|83] |. 

Using Tab. IVIIII we come to the following bounds on the codebook rates at decoder 1 : 

-^{i,2}^{i,2} + -^i^{i,2} + Li^i < I{Yi; ?7{i,2}^{i,2}, C^i^{i,2}, t^i^il, <3) (84a) 
ii^{i,2} < /(yi;?/i^{i,2},C/i^i|C/{i,2}^{i,2},Q) (84b) 
ii^i </(i^i;(7i^i|f/{i,2}^{i,2},C/i^{i,2},Q)- (84c) 
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TABLE IX 

Decoding error event table using Th. |V.8| for CIFC in \30g) for decoder 2 for Sec. |V-C31 







t^{l_2}->2 




Ul^2 


C/2^2 


p - 

^^-r 1 dcncodingS 


Dl 


X 










1 


D2 


/ 


X 








P 

^1 '^{1,2}^{1,2} 


D3 


/ 


/ 


X 




X 


p 

^1 '^{l,2>-*f 1,2> ''^-fl,2>-*2 


D4 


/ 


/ 


X 




/ 


p 

^1 '^{l,2f^{l,2l-''^-fl,2>^2''^2^2 


D5 


/ 


/ 


/ 


X 


X 


^l'^{l,2}^{l,2}>f^{l,2}-v2''^l-^{l,2} 


D6 


/ 


/ 


/ 


/ 


X 


p 

^l'^{l,2f^{l,2}'^{l,2>^2''^l^-{1.2>-'^1^2 


D7 


/ 


/ 


/ 


X 


/ 


p 

^l'^{l,2}^{l,2}'f^{l,2}^2''^l^{l,2}.'^2^2 


D8 


/ 


/ 


/ 


/ 


/ 


successful decoding 



Using Tab. IVIIII we come to the following bounds on the codebook rates at decoder 1 : 

< I(Yi; ?7{1,2}^{1,2}, f^l^{l,2}: t^{l,2}^2, t^2^2, t/l^2|Q) 

+ ^(C^i^{i,2}; C^2^{i,2}|t^{i,2}^{i,2}, Q) (85a) 

< I{Yi; Ui^[i,2}, U[i,2}^2, U2^2, Ul-¥2\U{i^2}^{l,2}iQ) 
+ I{Ul^{1.2}- C^2^{1,2}|C/{1,2}^{1,2}, Q) (85b) 

< I{Yi; ?7i^{i,2}, C^2^2, Ui^2\U[i^2}^{1.2}, C^{1,2}^2j Q) 
+ I{Ui^{l,2}; C^2^{1,2}|C^{1.2}^{1,2}, Q) (85c) 

< I{Yi; ?7l^{l,2}, t^l^2|t^{l,2}-.{l,2}, t^{l,2}^2, U2^2, Q) 
+ I{Ul^{1.2y. C^2^{1,2}|C/{1,2}^{1,2}, Q) (85d) 

< I{Yi; Ui^2, (72^2|(7{i,2}^{i,2}, (7{i,2}^2, C/i^{i,2}, Q) (85e) 

< /(Fi; ;72^2|(7{l,2}^{1.2}, t/{1.2}^2, C^l^{1.2}, Ui^2,Q) (85f) 

</(ri;;7i^2|(7{i.2H{i.2},f/{i.2}^2,C/i^{i,2},C/2^2,Q). (85g) 

The term /(C/i-+{i,2}; '^2-»{i,2}|t^{i,2}-»{i,2}) can be interpreted as the improvement on the codebook rates provided by 
binning: since 2} < C^2^{i,2}i the decoder looks for two codewords U^_^^^ 2}; ^2->{i 2} ^^^^ the typicality properties 

imposed by the encoding procedure. This is an extra requirement on the decoded codewords that makes the decoding errors 
less likely. This term is present only when either or both the two codewords are incorrectly decoded, since this advantage is 
present only when one of the two codewords is incorrect. 

Note that Tab. IVIIII and |IX] are very similar to Tab. |VT] and Tab. IVIII given the similarity in the proofs of the two theorems, 
the codebook bounds are derived in a very similar manner In the following we will use only one table to represent both cases 
by simply adding the last column of one graph to the other. 

Remark V.13. When designing achievable schemes, one often rate splits a message into common and private parts, imposing 
the decoding of part of a user's message to another user who is not interested in this common message. In this case the wrong 
decoding of the rate-spUt common message does not represent a decoding error for the non intended user From this it follows 
that some of the error events in Th. IV. 8 1 and in Th. IV.l II are not actually error events for some of the users. In this specific 
instance the bounds of the codebook rate are no longer valid and the overall region larger. Our model does not cover this case 
but such occurrence can easily be taken into consideration when deriving achievable schemes for specific channel models. 

D. Achievable region 

We can finally combine the results in Sec. IV.4I Sec. IV.6I Sec. IV-C2I and Sec. IV-C4I to come to the main theorem of this 
paper. 

Theorem V.14. Achievable region For any CGRAS a rate vector R is achievable if there exists a time-sharing RV Q as 
defined in (|9]l, a rate-splitting matrix T as defined in (III-El l. and a binning rate vector R such that R satisfies the constraint 
in Th. \V.4\ or Th. \V.6\ and L = R' + R satisfies the constraint in Th. \V.8\ or Th. \V.11\ 

VI. Achievable schemes for classic multi-terminal networks 

We now re-examine the multi-terminal networks in Tab. |lll] and re-obtain some of the achievable region involved in the 
capacity results in Tab. H] The purpose of this section is twofold: First, we show how the CGRAS provides an important tool 
to quickly derive achievable schemes for a large class of channel; by carefully tabulating the error events and making a few 
simple calculations, one is able to determine an achievable region for the channel under study. Second, the CGRAS provides 



-^{1,2}^{1,2} + -^1^(1,2} + -^{1,2}^2 + -^2^2 + -^1^2 

il^{l,2} + i{l,2}^2 + -^2^2 + Li^2 

Ll^{l,2} + -^2->-2 + Li^2 

+ Li^2 

Ll^2 + L2^2 
L2^2 
Ll^2 
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an important tool to explore the possible transmission strategies available on a channel. Given the distribution of the message 
and the channel model, one immediately sees how to apply superposition coding and binning among the different codewords. 
Both of these observations about the CGRAS makes it a very important tool to approach a number if information theoretical 
problems. In this section we also hope to familiarize the reader with the construction of the CGRAS and of the derivation of 
the rate bounds. 



A. Multiple access channel with common information 

The Multiple Access Channel (MAC) is a three-terminal network where two transmitters want to each communicate a 
message to a decoder In this section we consider the Multiple Access Channel with Common Messages (MAC-CM), where 
the transmitters send an additional message, known at both receivers, to the receiver. This model is obtained from the from 
the general interference channel when C ({i?i^i, i?2^i, ^{i,2}->i})^ that is when the rates different from i?i^i,i?2^i and 
R{i,2]^i are all set to zero. The capacity of MAC was determined by Ahlswede |12| and Liao I1I6J while the capacity of the 
case with common messages was shown by Slepian and Wolf |43|. 

Since there is only one receiver, it is possible to split the message PVi^m only in a sub-message of the form Wj'!,^^' . The 
set of all possible 1 is {1,2, {1,2}} and thus rate-splitting does not increase the number of messages transmitted over the 
channel. 

Without loss of generality we can define the matrix Fmac-cm as 

I- a 
1 




Tmac- 



-CM 



/3 



a 
1 - 



so that (III-El i becomes 









' (1 




"-R{1,2}^1 








(1 


-/3)i?2^1 + 


"-R{1,2}^1 




^{1.2)^1 








(1 - a - /3)i?{i,2}^l 



(86) 



By considering all the possible choices of G [0, 1]^ one obtains all the possible rate-splitting choices. Taking the union 

over all the rate-splitting matrices for an achievable region TIr/ vector produces the convex hull i?convcx(-AR'). 
Given the distribution of the messages, the possible edges in 5 and B are; 



U2- 

U2- 



■l<t^{l,2}^l, 
■l<t^{l,2}-+l: 
1 < t^{l,2}^l7 
1 < U{1,2}^1- 



(87a) 
(87b) 
(87c) 
(87d) 



One can now obtain a number of achievable regions by combining these edges in dSTl i in different ways. Here, and also for 
the following examples, we focus on three transmission strategies: one employing only superposition coding, one only binning, 
and one involving both. More specifically: 

1) Superposition coding scheme: 



[/l^l<C/{l,2}- 
t/i^2<C/{l,2}- 



(88a) 
(88b) 



2) Binning based scheme: 



Ui^l < C/{1_2}^1, 
Ui^2 < C/{1,2}^1, 



(89a) 
(89b) 



3) Binning and superposition based scheme: 



J7i^i<?7{i,2}-,i, 

C/2^1 < ?7{1,2}^1- 



(90a) 
(90b) 



In each scheme we have 



encoding — -Poncoding " -^Q -Pc/{i,2}^i 1 (7{i,2}-.i .Q-Pt^i^ i |C^{1,2}^1,Q (91) 

The three schemes are depicted in Fig. |7] the convention used to represent the CGRAS is the same as in Fig. |4] Fig. |5] and 
Fig. |6l the only variation here is that we have added violet rectangles to identify the common message. 
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^{1,2}- 



1 



Ui. 




1 



(a) Superposition coding based scheme. 
Fig. 7. Different achievable schemes for the MAC-CM channel. 



(b) Binning based scheme. 



(c) Binning and superposition based scheme 



TABLE X 

Decoding error of event table for the CGRAS in (88), also pictured in Fig. |7(a)| 





'^{1,2}^! 




U2-.1 


p ,_ 

yils 


Dl 










D2 


/ 


m 




-''nic/fi,2}-.i 


D3 


/ 


/ 




-Py-iI!72^1.!^{1,21^1 


D4 


/ 


m 


/ 





1} Superposition coding based scheme for the MAC-CM: 

Let's consider first the scheme with only superposition in ( |88] l. also pictured in Fig. |7(a)| Since there is no binning edge, the 
graph representing the achievable scheme is an ADG and thus we indeed have CGRAS. Also, since no binning is performed, 
encoding is always successful as Pcncoding = ^codcbook and we can set = 0, V(i, j). 

As we have seen in Sec. IV-CI that a single table can be used to represent the bounds in both Th. IV.4I and Th. IV. 6 1 Tab. 
IVI-All is the single tabl e for the CGRAS in dMJl- 

With the help of Tab. IVI-All we conclude that the achievable region with successive decoding is 

i?{i,2Hi </(yi;C/{i,2Hi|Q) (92a) 
Ri^i < /(Yi;t/i^i|C/{i,2}^i,Q) (92b) 
R2-.1 < /(Yi;t/2^i|C/(i,2Hi,g), (92c) 

while the achievable region with joint decoding is 

R{i.2}^i + Ri^i + R2^i < I{Yi; (7{i,2}^i, Ui^i, U2^i\Q) (93a) 

Ri^i < /(ri;;7i^i|(7{i,2}^i,Q) (93b) 

R2^i < I{Yi;U2^i\U^,^2}^„Q), (93c) 

for any distribution that factors as ( |9TT l- 
Note that 

= I{Yi- t/{i,2}^i|g) + I{Yu Ui^i, C/2^i|C/{i,2}^i, Q) (94a) 

= /(yi;t^{i.2}^ilQ) + /m;(7i-i|c/{i.2Hi,Q) + /m;C/2^i|c/{i,2Hi,0) (94b) 

= + + ^M, (94c) 

so the achievable region with joint decoding includes the achievable region with sequential decoding for any fixed codebook 
distribution (l9Tl i. 

2) Binning based scheme for the MAC-CM: 

Let's consider the scheme with only binning in (l89l l, also pictured in Fig. |7(b)| The graph of this achievable scheme is a 
UDG thus we indeed have CGRAS. For this scheme 



C/2^l|Q' 



and thus 



|-cncoding 

codebook 



/((7{i.2}^i; C/1-.1IQ) + /(C/{i,2}^i; U2^i\Q) 



(95) 



(96) 



The encoding fails when it is not possible to determining the bin indexes bi^i and 61^2 such that Ui^i{wi^i,'Wi^i) and 
f^2-i(w2^i7 «^2^i) appear to be generated conditionally dependent from C/{i^2}^i although generated independently from this 
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TABLE XI 

Encoding error of event table for the CGRAS in {89), also pictured in Fig. |7(b)| 









distribution for Tli. |V.4| 


JM.U.L. foj. Th.IV.6l 


El 


Kl 


m 


■Pt/{l,2>-l-Pyi^l-Py2^1 





E2 


/ 




■^'^{l,2}-l-^tfl-l-Py2^l|C/{i,2}^l 


/(C/2^i;C/{i,2}^i[Q) 


E3 


Kl 


/ 


■Pt' { 1 , 2 } ^ 1 -^tr 1 - 1 1 C/{ 1 , 2 } - 1 -^^£^2 1 


7((yi_i;L/{i,2}^i|g) 



codeword. These two bin indexes are determined independently from each other, so we do not expect any difference between 
the rate bounds obtained with Th. IV.4I and Th. IV. 6 1 

As pointed out in Sec. IV-CI a single table can be used to represent the bounds in both Th. IV.8I and Th. IV. Ill Tab. |Xl] is the 
single table for the CGRAS in (|89] |. 

With the help of Tab. IXIII , we have the binning rates bound obtained by both Th. IV.8I and Th. IV. Ill which are: 

Ri^i + i?2^i > /(C/{i,2}^i;(7i^i,t/2^i|g) (97a) 
fli^i >/(C/{i,2Hi;f/i-.i|Q) (97b) 
i?2^i >/(C/{i,2Hi;C/2^i|Q). (97c) 

The decoding error table is for this achievable scheme is again Tab. IVI-All The bounds on the codebook rates that we 
obtain from Tab. IVI-All are: 

L{i,2}^i </(yi;f/{i,2}^ilQ) (98a) 
^1^1 </(i"i;(7i^i|C/{i,2Hi,g) + /(C/{i.2}^i;C/i^i|0) (98b) 
^2^1 </(ri;C/2^i|C/{i,2Hi,g) + /(C/{i,2}^i;C/2^i|Q), (98c) 
while using joint decoding we have: 

i{i,2}^i + Li^i + Li^2 < (7{i,2}^i, Ui^i, t/2^1) + /([/{i,2}^i; C/2^i|0) (99a) 

ii^i </(i"i;(7i^i|C/{i.2Hi,g)+/(C/{i,2Hi;C/i^i|Q) (99b) 
i2^i </(i"i;C/2^i|C/{i.2}^i,g) + /(C/{i,2Hi;C/2^i|g). (99c) 

As for ( |94l i. the region obtained with joint decoding is larger than the one for sequential decoding. 

In order to obtain the rate vector R from L and R we can use the Fourier-Motzkin elimination procedure. The Fourier- 
Motzkin Elimination (FME) (44) is for eliminating variables from a system of linear inequalities. The FME is a crucial tool 
to obtain achievable regions for multi-terminal networks as it allows one to obtain an expression of the achievable region in 
the message rates R from a region expressed using the rate-splitting rates R. The FME of the codebook bounds on L using 
joint decoding in (|99l l and the region on the binning rates (|97| i is 

^{i,2}^i + Ri^i + R2^i < I{Yi; ;7{i^2}^i, Ui^i, U2^i\Q) (100a) 
Ei^i < /(Yi;;7i^i|(7{i,2}^i,g) (100b) 
i?2^i </(yi;;72^i|t/{i,2}^i,g), (100c) 

which is the capacity region. 

3) Superposition coding and binning based scheme for the MAC-CM: 

Let's consider now the scheme with only superposition in (|90] l, also pictured in Fig. |7(c)| Since there is no joint binning in 
this scheme, the graph representing the achievable scheme is an ADG and thus we indeed have CGRAS. 

Since there is only one bin index to determine 62^1. there is only one error event and there is no difference between the 
encoding error analysis between Th. IV.4I and Th. IV.6I which both yield the bound: 



i?2^i > /(C/2^i;(7{i^2}^i|g). 

Again Tab. IXIII helps us derive the codebook rate for the sequential decoding case: 

i?{i.2}^i C/{i,2Hi) (lOla) 

Ri^i > /(>l;f/i^i|C/{i,2Hi) (101b) 

L2^i > /(ri;{/2^i|[/{i,2Hi), (101c) 
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as well as the codebook bounds for the joint decoding case: 



i?{i,2}^i + Ri^i + Li^2 < I{Yi; C/{i,2}^i, Ui^i, C/2^1) + /(?7{i,2}^i; U2^i) 

</(yi;C/i^i|;7{i.2}^i) 

L2^i < I{Yi] C/2^i|C/{i,2}^i) + I{U{i.2}^i; U2^i), 
The Fourier-Motzkin elimination of the encoding and decoding bound yields 

^{1,2}^1 + ^1^1 + R2^1 < ^(^i; ^^{1,2}^!: Ui^i, C/2^1) 

-Ri^i </(l^i;(7i^i|t/{i,2}^i) 

R2^1<I{Yl■,U2^l\U^,^2}^,), 

which is again the capacity region. 



(102a) 
(102b) 
(102c) 
(102d) 

(103a) 
(103b) 
(103c) 



In this section we have seen that superposition coding, binning, and a mixture of the two strategies, achieve capacity for 
the MAC channel. This result is somewhat surprising giving the different nature of the encoding schemes. Note that proving 
capacity using binning requires the use of the FME, which becomes more and more complex as the number of binning indexes 
increases. 



B. Broadcast channel with common messages 

The Broadcast Channel (BC) is a three-terminal network where one receiver communicates two messages to two de- 
coders. In this section we consider the Broadcast Channel with Common Messages (BC-CM), where the receiver sends 
an additional message to be decoded at both receivers. This model is obtained from the general interference channel when 
6 ({i?i^i, i?i^2, ^i^{i,2}})> that is, when the rates different from i?i^i,i?i^2 and i?i^{i,2} are all set to zero. 

The broadcast channel was first studied by Cover in 1972 |30| . The largest region for the general BC channel is derived 
by Marton |37| and represents capacity for the degraded BC ||30l . Q, IS), the deterministic BC ||45l . the less noisy BC, and 
the more capable BC |18|. For the degraded BC, the less noisy BC, and the more capable BC, superposition coding achieves 
capacity while binning achieves capacity for the deterministic broadcast channel. 

As for the MAC-BC, rate-splitting does not increase the number of messages to be transmitted over the channel: since there 
is only one decoder, W^i^j can be rate-spitted only in messages of the form Wl^y Moreover there are only two decoders, so 
1 can only take values in {{1, 2}, 1, 2}. 

Without loss of generality we can define the matrix Fbc-cm as: 



so that jll-EI ) becomes 







' 1 - 


- a 


a 




rsC-CM — 









7 


















R'l^i 




" (1 


- a)Ri^i + 


aR{i,2}~*i 


R' = 






(1 


-p)Ri^2 + 


aR{i,2}^i 




Rl^{l,2} 








(1 - a - /3)i?i^{i,2} 



(104) 



By considering all the possible choices of {a,f3) e [0, 1]^ one obtains all the possible rate-splitting choices. Taking the union 
over all the rate-splitting matrices for an achievable region TIr/ vector produces the convex hull i?convex (-Ar' ) . 
Given the distribution of the messages, the possible edges in S and B are: 



C/l^l<[/i^{1^2} 


(105a) 


Ui^2<Ui^{l^2} 


(105b) 


Ui^l < Ui^{i^2} 


(105c) 


Ui^2 < C^l^{1^2} 


(105d) 


Ul^{1^2} < Ui^i 


(105e) 


Ul^{1^2} < Ui^2- 


(105f) 



By means of the general approach to the transmission scheme design provided by the CGRAS, we determine alternative 
transmission schemes to the Marton achievable scheme. As for Sec. IVI-AI we focus on three transmission strategies: one 
employing only superposition coding, one only binning, and one involving both. 
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1) Superposition coding scheme: (Marton scheme) 



?7i^i<?7i^{i^2} 

?7l^2<C^l^{l,2} 



(106a) 
(106b) 
(106c) 
(106d) 



2) Binning based scheme.: 



Ui^l < C/l^{l,2} 
Ui^2 < t^l^{l,2} 

Ul^2 < Ui^i 
Ul^l < Ui^2, 



(107a) 
(107b) 
(107c) 
(107d) 
(107e) 
(107f) 



3) Binning and superposition based scheme: 



Ui^l < t/l^{l,2} 
Ui^2 < C^1^{1,2} 
Ul^{1.2}<Ul^l 
Ul^2 < Ul^l 



(108a) 
(108b) 
(108c) 
(108d) 
(108e) 



In each scheme we have that 



with 



and 



^encoding — ^Q-P(7i^{ i_2} C^l-l C/l^2 |Q ' 



pi _ p p 

encoding Q Ui^{i2}Ui^i\Qi 



ding — ^Q-Pc/i^{i ai^^i- 



(109) 
(110) 
(111) 



The three schemes are depicted in Fig. |VI-Bl the convention used is the same as for the previous representations of CGRASs. 






^1-{1,2} 



(a) Superposition coding based scheme. (b) Binning based scheme. 

Fig. 8. Different achievable schemes for the BC-CM channel. 



(c) Binning and superposition based scheme. 
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TABLE XII 

Encoding error of event table for the CGRAS in (88), also pictured in Fig. |8(a)| 





Ui^i 




distribution for Th. |V.4| 


JM.U.L. foj. Th.IV.6l 


El 






-ff^l-,{1.2}-Pf/l-,l|C/l^fl,2}-^t'2^l|Cl^fl,2> 





E2 
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-''f^l { 1 . 2 } C'l ^ 1 1 C/l ^ { 1 , 2 1 ■Pt'l - 2 


/(C/i_i;l/i^2|C/i->{i,2},Q) 


E3 
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1 , 2 } ^ 1 C^l 1 2 1 C/ { 1 , 2 } ^ 1 


/(C/i^i;C/i^2|C/i^{i,2},Q) 



TABLE XIII 

Decoding error of event table for the CGRAS in (88), also pictured in Fio. |8(a)| 





'^1^{1,2} 




p — 


Dl 








D2 
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PyiIu^i^2}^i 



1) Superposition coding based scheme for the BC-CM: 

Let's consider first the scheme with only superposition in (1106b . also pictured in Fig. |8(a)| which corresponds to the Marton 
scheme for the BC in [37|. This scheme satisfies the condition of Th. IIII. 1 1 and is thus a CGRAS. The equivalent ADG is 
obtained by choosing an orientation for the undirected edge Ui^i < >Ui^2- 

This scheme has two bin indexes 61^1 and 61^2 and the possible encoding error events can be found in Tab. IXIII 

The encoding error analysis using Th. IV.4I yields the bounds 



+i?i^2 > /(C/i^i;C/i^2|C/i^{i,2},Q) (112a) 

i?i^2>0 (112b) 

i?i^i>0, (112c) 

while Th. IV.6I gives the binning rate bounds 

i?1^2+i?1^2 >/(C/l^i;«7l^2|C/l^{l,2},Q), (113) 

with 

i:oTebrk-^(c^i^i;c^i^2|c/i.{i,2},g). 

Therefore, as for the achievable regions for MAC in Sec. IVI-AI the two theorems yield the same bound on Ri^i and Ri^2- 
The table representing the possible decoding errors at decoder 1 is Tab. IXIIII since the two receivers are symmetric, we only 
need to derive the codebook bounds for one decoder The bounds for the other decoder can be obtained by proper rearrangement 
of the indexes. 

The codebook rate bounds obtained with successive decoding at decoder 1 are 

i?i^{i,2} </(n;t/i^{i,2}|Q) (114a) 

Li^i < /(yi;C/i^i|[/i^{i,2},Q) (114b) 

i?i^{i,2} </(r2;[/i^{i,2}|Q) (114c) 

Ll^2 < /(>2;C/i^2|C/i^{i,2},Q), (114d) 

while with joint decoding we instead obtain 

i?i^{i,2}+ii^i </(n;C/i^i,C/i^{i,2}) (115a) 

Li^i < /(yi;i7i^i|i7i^{i,2}) (115b) 

i?i^{i,2}+ii^2 </(1^2;C/i^2,C/i^{i,2}) (115c) 

il^2 </(>'2;C/l^2|C/l^{l,2}). (ii5d) 

Again, joint decoding outperforms sequential decoding. 
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TABLE XIV 

Encoding error of event table for the CGRAS in U07K also pictured in Fig. |8(b)| 
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{/i^{i,2} ) + /(i7i->2; C7i_{i,2} ) 
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-PC^1^{1,21.'^1^1'PC^1^2 
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By FME of the binning rate bound, and the codebook bounds obtained by joint decoding, we obtain the region: 

</(n;t/i^i|C/i^{i.2},Q) (116a) 

Ri-,2 < I{Yi;Ui^2\Ui^{i,2},Q) (116b) 

+ Ri^2 < I{Yi; ;7i^i|C/i^{i,2}) + I{Yi; Ui^2\Ui^[i^2}) - HUi^i; Ui^2\Ui^{i,2},Q) (116c) 

i?i^{i,2}+i?i^i t/i^i,t/i^{i,2}|Q) (ii6d) 

i?i^{i,2}+i?i^2 </(l2;C/i^2,C/i{i,2}|Q) (116e) 
2i?.i^{i^2} + Ri^i + i?i^{i,2} < I{Y,: (7i^{i,2}|Q) + 1(^2; C/1^2, (7i{i,2}|Q) - /(t/i^i, ;7i^2|C/i->{i,2}, Q) (116f) 

Rl^l + i?l^{l,2} + i?1^2 < HYi: C/l^l|C/l^{l,2}, Q) + /(>'2; C/1^2, C/l{l,2}IQ) - /(C/1^1, C/l^2|C/l-.{l,2}, Q) 

(116g) 

+ i?i^{i^2} + ^1^2 < I{Y2; C/i^2|C/i^{i.2}) + C/i{i,2}) - /(C/1^1, C/i^2|C/i^{i,2}), (116h) 



It now appears clear that there is a difficulty in applying the FME to obtain an achievable region expressed as a function 
of R. When CGRAS contains even a small number of binning rates, the FME produces a region containing a large number 
of equations that can no longer be easily manipulated. In particular, it becomes increasingly difficult to match inner and outer 
bound expressions, especially when the coefficients multiplying the rate bounds are different from one. 

2) Binning based scheme for the BC-CM: 

Let's consider the scheme with only binning in ( I107l i. also pictured in Fig. |8(b)| This scheme has three binning indexes and 
three steps of joint binning: the graph representing the achievable scheme is a UDG and thus it is also a CGRAS. It is easily 
shown that the equivalent ADG is obtained by choosing any orientation for the undirected edges that does not form a cycle. 

Tab. IXIVI helps us determine the binning rate bounds. Using Th lV.4l we obtain the bounds: 

i?i^{i,2} + Ri^i + Ri^2 > /(C/1^1; Ui^2\Q) + I{Ui^2- Ui^i, C/i^{i^2}IQ) (117a) 

i?i-.{i,2} + i?i^i >/(c/i^i;f/i^{i.2}IQ) (117b) 
i?Wi,2} + i?i-2 >/(C/i->2;{/i^{i.2}|Q) (117c) 

:ri^i+:ri^2 >/([/i^i;f/i^2io) (ii7d) 

while using the mutual covering lemma of Th. IV.6I we obtain 



since 



0>0 (118a) 

Ei_»i>0 (118b) 

i?i^2>0 (118c) 

i?i^i+i?i^2 >/(C/i^i;C/i^2|0) (118d) 

i?i^{i,2}>0 (118e) 

i?i-.{i,2} + i?i-2 >/(c/i^{i,2};(:^i-2|Q) (ii8f) 

i?l^{l,2}+i?1^2 >/(f/l^i;f/l^{l,2}|Q) (118g) 

i?i^{i,2} + + i?i^2 > /({/i^i; f/i^{i,2}|Q) + /(C/1^2; ;7i^{i,2}|Q), (118h) 

Cd^boSc = iiUi->i; Ui^{i,2}\Q) + /(C/1-.2; c/i^{i,2}IQ), (119) 
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TABLE XV 

Encoding error of event table for the CGRAS in U08K also pictured in Fig. |8(c)| 





^1— )-| 1 2} 






Distribution for Th. |V.4| 


JM.O.L. foj Th IV 61 


El 






m 




/{C/i^i;C/i^2,C/i-,{i,2}l<3) 


E2 


m 


m 


/ 


■Pt'l ^ 1 1 LTl ^ 2 -Pc/i ^ 2 , C7i ^ { 1 _ 2 } 


7'(C/i-.i;C/i^{i,2},l/i^2|Q) 


E3 


m 


/ 


m 


■P'^l^{l,2}>Lri^l>Lri^2 


7-((7i_i;(7i^2,C/i^{i,2}IQ) 


E4 


m 


/ 


/ 


■^fl^fl,2}.C^l-l>Lri^2 


7((7i^i;t;i_{i,2}|L'i-.2,g) 


E5 


/ 


m 


m 


■Pc/l_i|i7i_{i_2}-fUl^2,fl^{l,2} 


/((7i^i;C/i^{i,2}|Q) 


E6 


/ 




/ 


■Pfi-n,2}>ui^i(7i^2 


/(C/i-.i;(7i-.2,(7i^{i,2}|Q) 


E7 


/ 


/ 


m 


■f'fi-n,2}>ui^2,c7i^i 


/(C/i^i;(7i^2|t/i^{i,2},Q) 


E8 


/ 


/ 


/ 


■P'^l^{l,2>.C^l^l>t^l-2 






Now for the decoding operation: again we only consider the error events at one decoder, since the two decoders are symmetric. 
Consider for instance decoder 1 that decodes J7i^{i^2} ™d Ui^i. For decoder 1 it is not possible to incorrectly decode one 
of the two codewords if the other is incorrectly decoded, since the binning index of one codeword depends on the other and 
vice versa. For this reason only two events are possible: either both codewords are correctly decoded or incorrectly decoded. 
For this reason both Th. IV.8I and Th. IV. 1 II result in the codebook bounds 

ii^{i,2} + Li^i < HYi; L/i^i, ;7i^{i,2}) + I{Ui^{u2}; Ui^i) (120a) 

il^{l,2} + ^2^2 < nY2; Ui^2, (7i^{i,2}) + /(f/l^{l,2}; Ui^2), (120b) 



From the FME of the binning rates we obtain the region 

Ri^{ia} + Ri^i < I{Y,;Ui^uU,^[,^2}\Q) (121a) 

i?i^{i.2} + i?i^2 < /(l2;{/i^2,;7i^{i,2}|Q) (121b) 
2i?i^|i,2} + Ri^i + Ri^2 < I{Yi;Ui^i, (7i^{i,2}|Q) + 1(^2; C/1^2, C/i-.{i,2} IQ) 

-/(C/i^i;C/i^2|C/i^{i,2},Q), (121c) 

while from the FME of the rate-splitting we again obtain Marton achievable region. 
3} Superposition coding and binning based scheme for the BC-CM: 

Let's consider the scheme with only binning in ( 11081 ), also pictured in Fig. |8(c)| This scheme has three binning indexes and 
two steps of joint binning; the graph representing the achievable scheme is a UDG and thus it is also a CGRAS. It is easily 
shown that the equivalent ADG is obtained by choosing any orientation for the undirected edges that does not form a cycle. 

The encoding error events are summarized in Tab. IX VI The two bounds for the binning rates obtained with Th. IV.4I are 



i?i^n,2} + + i?i^2 >/(C/i^i;C/i^2,C/wi.2>IO) (122a) 

(122b) 
(122c) 
(122d) 
(122e) 
(122f) 
(122g) 
(122h) 



} + Ri^i - 




.2 >/(C/i^i;C/i^2,C/i^{i.2}IO) 


^1^(1,2} - 




.1 >/(c/i^i;c/i^{i,2}i;:^i-2,g) 


^1^(1,2} - 




.2 > 


^1^(1,2} - 




.2 > 


Rl-*l ' 




.2 >/(C/l^i;C/l^2|C/l^{l,2},Q) 




Ri- 


.1 >o 




Ri- 


.2 > 






> 



while the ones obtained with Th. IV.6I are: 



> (123a) 

Ri^2 > (123b) 

Ri^i > (123c) 

+ Ri^2 > HUi^i; f/1^2) (123d) 

i?i^{i,2} > HUi^i; Ui^2\Ui^{u2}) (123e) 

Ri^{i.2} + Ri^2 > (123f) 

i?i^{i,2} + Ri^i > HUi^i; C/i^{i,2}) (123g) 

i?i-.{i,2} + +i?i^2 > /(C/i^i;C/i^2,C/i^{i,2}|0) (123h) 
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TABLE XVI 

Decoding error of event table at decoder 1 for the CGRAS in (T08j, also pictured in Fig. |8(c)| 







Ul^2 


Py^jg at decoder 1 


El 




X 




E2 


/ 







with 



j-cncociing 

codcbook 



/(C/i^i;C/i^2,C/i^{i,2}IQ). 



(124) 



We now see that the two theorems provide different bounds on the binning rates and that it is no straight forward to reaHze 
which of the two regions is smaller. 

The decoding error at receiver 2 is summarized in Tab. IVI-B3I through which we obtain that the bounds using successive 
decoding are 

Li^{i^2} </(>^2;C/i^{i,2}IQ) (125a) 
Li_i < /(r2;{/i-»2|C/{i,2}^i|Q) (125b) 

while with joint decoding we have 

^1^(1,2} + Li^2 < I{Y2. Ui^2, Ui^[,,2}\Q) (126a) 
Li^2 < I{Y2; (7i^2|C/{i,2}^i, Q) (126b) 
For decoder 1, we have the same decoding errors at decoder 1 in Sec. IVI-B2I 

ii^{i.2}+ii^i c/i^i,c/i^{i,2}|g) + /(t/i^{i,2};f/i^i 10) (127) 

The FME of the binning rates obtained with the covering lemma and the codebook rates obtained with joint decoding gives 



Ri- 

Ri- 
Ri- 



Ri^2 < I{Y2; ;7i^2|C/{i,2}^ilQ) (128a) 

i?i^{i,2} < I(Yi;Ui^i, C/{i,2}^i|Q) (128b) 

RiMi,2} < I{Y2] Ui^2, C/{i,2}^i|0) (128c) 

Ri^Ha} < HYuUi^,, C/{i,2}^i|g) + HY^; C/i^2|C/{i,2}-.i, Q) - HU^^i; Ui^2\Ui^{i.2}) (128d) 

i?i^{i,2} < /(n; C/i^i, C/{i,2}^i|0) + HY2; C/1^2, C/{i,2}^i, 0) - nUi^i; C/i^2|C/i^{i,2}) (128e) 



C. Interference channel 

The InterFerence Channel (IFC) is a four-terminal network where two pairs of transmitter/receiver pairs want to communicate 
a message over the channel. This model is obtained from the general interference channel when 6 ({i?i^i, i?2^2})i that is, 
when the rates different from and i?2-»2 are all set to zero. 

The IFC was first proposed by Shannon |[3T| and the first capacity results are due to Ahlswede ||32ll . Capacity is known in 
the strong interference regime, 1461 . ||47]| . pS], that is, when there is no loss of optimality in having both decoders decode 
both messages. The capacity for the deterministic case was determined in [22 1. The largest known inner bound is by Han and 
Kobayashi 111] and employs rate-splitting and superposition coding. Unlike for the BC-CM and the MAC-CM, rate-splitting 
increases the number of messages for the IFC. We can rate-split each message into common and private parts, i.e. Wi^i can 
be rate-split into W[^i and W^^^j^ 2}' and similarly, W2.2 can be rate-split into W2^2 and W^^^^ 2}- This rate-splitting can 
be represented using the matrix Fipc 

Without loss of generality we can define the matrix Fbc-cm as 



IFC 



a 





1 — a 













1-/3 



so that dll-El l becomes 



R' 



r ^'1- 


+1 




ai?i^i 


Rl 


-{1,2} 




(1 - a)i?i^{i,2} 


R'2- 


*2 




/3i?2^2 




-{1,2} . 




_ (1 - /3)i?i^{i,2} 



(129) 
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TABLE XVII 

Decoding error of event table for the CGRAS in (TIT), also pictured in Fig. |9(a)| 











Dl 


m 






D2 


/ 


m 





By considering all the possible choices of (a,/3) G [0,1]^ one obtains all the possible rate-splitting choices. Taking the 
union over all the rate-splitting matrices for an achievable region Au' vector produces the convex hull i?convox(-^R')- 
Given the distribution of the messages, the possible edges in 5 and B are: 



t/i^l<C/i^{l_2} 


(130a) 


t^2^2<C^2^{l,2} 


(130b) 


Ul^l < ?7i^{i_2} 


(130c) 


t/2^2 < C^2^{1,2} 


(130d) 




(130e) 


C^2^{1,2} < C^2^2- 


(130f) 



By means of the general approach to the transmission scheme design provided by the CGRAS, we determine transmission 
schemes alternative to the Han and Kobayashi achievable scheme As for Sec. IVI-AI we focus on three transmission strategies: 
one employing only superposition coding, one only binning, and one involving both. 

1) Superposition coding scheme: (Han and Kobayashi scheme) 

f/i^i<?7i^{i,2} (131a) 
t/2^2<C/2^{i,2}, (131b) 



2) Binning based scheme.: 

< C/i^{i,2} (132a) 

C/i^{i,2} < Ui^i (132b) 

U2^2 < t^2-.{l,2} (132c) 

U2Ml,2} < U2^2, (132d) 



3) Binning and superposition based scheme: 

C/i^i<C/i^{i.2} (133a) 

C/2^2 < t^2^{l,2} (133b) 

;72^{l,2} < U2^2- (133c) 



In each scheme we have 

-Pcncoding = PqPUi^iUi^^i_2}\qPu2^2U2^ii,2}\Q (134) 

while 

^encoding = PqPui^iUi^ii_2}\Q (135) 

and, in a symmetric fashion, 

^encoding = PqPu2^2U2^ii,2}\Q (136) 

The three schemes are depicted in Fig. IVI-CI the convention used to represent the CGRAS is the same as for the previous 
figures representing a CGRAS. 
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2^2 



\Ui- 



u, 



2^2 



2^2 



■{1,2} 



t/9 



■{1,2} 



.^1-{1,2} 



■{1,2} 



(a) Superposition coding scheme. 
Fig. 9. Different achievable schemes for the IFC channel. 



(b) Binning based scheme. 



(c) Binning and superposition based scheme. 



TABLE XVIII 

Encoding error of event table for the CGRAS in jl06> . also pictured in Fig. |8(a)| 





t^l-»{l,2} 




distribution for Th. IV.4I 


jM.c.L. forTh.IV.6l 


El 











E2 




/ 




/(C/i^i;(l/i^{i,2}) 


E3 


/ 






/(C/i^i;(l/i^{i,2}) 



1 ) Superposition coding based scheme for the IFC: 

Let's consider first the scheme with only superposition in (1131b . also pictured in Fig. |9(a)| which corresponds to the Han 
and Kobayashi scheme for the IFC in ifTTI . Since this scheme involves only superposition coding, its graph representation is 
a ADG which makes in a CGRAS. 

The table summarizing the error event at decoder 1 is Tab. IVI-Cll Using successive decoding of Th. IV. 8 1 we have the 
decoding error event bounds: 

i?i^{i,2} C/i^{i,2}) (137a) 

Ri^i > /(ri;[/i^i|C/i^{i.2}) (137b) 
^2^(1,2} >/(1^2;C/2^{i,2}) (137c) 

i?2^2 > IiY2; U2^2\U2^{1,2}), (137d) 

while with the joint decoding of Th. IV. 1 1 1 we have: 

i?i-.{i.2} + Ri^i < I{Yi; (7i^{i,2}, C/i^i) (138a) 
Ri^i < I{Yi; Ui^i\Ui^{i,2}) (138b) 
i?2^{i.2} + i?2^2 < I{Y2; U2^{i,2}, f/2^2) (138c) 

i?2^2 < I{Y2; U2^2\U2^{1,2})- (138d) 

The classic Han and Kobayashi region for the IFC is obtained by considering the FME using the equality in ( I129l l. It was 
later shown that some of the rate bounds can be dropped from the FME |49|, thus resulting 

(139a) 
(139b) 
(139c) 
(139d) 
(139e) 
(139f) 
(139g) 





Ri 


< 




Ul^l,U,^[l,2}\U2- 


^{1,2}, Q) 






i?2 


< 


I{Y2 


U2^2, ^^"2^(1, 2} t^l- 


^{1,2}, Q) 




Ri 


+ i?2 


< 


I{Yi 


Ui^l, J/l^{l,2}, U2 


-^{i,2}l<3) - 


h/(r2;;72^2|t/2^{l,2},t/l^{l,2},Q) 


Ri 


+ i?2 


< 


I{Y2 


U2^2, U2^{1,2}, Ui 


-.{1.2}|Q)- 


h/(Yi;;7i^i|[/i^{i,2},C/2^{i,2},Q) 


Ri 


+ i?2 


< 


I{Yi 


Ul^l, U2^{1.2}\Ui^{i,2},Q) - 


^/(i"2;(72^2,(7i^{l,2}|t/2^{l,2},Q) 


2Ri 


+ i?2 


< 




Ul^l, ?/l^{l,2}, U2 


^{1,2}|0)- 


F/(i"2;C/2^2,C/l^{l,2}|C/2^{l,2},Q) 


Ri 4- 


2i?2 


< 


I(Y2 


U2^2, t^2^{l,2}, Ui 


^{1,2}IQ)- 


H/(ri;C/i^i,c/2^{i,2}|c/i^{i,2},Q) 



2) Binning based scheme for the IFC: 

Let's consider the scheme with only superposition in ( I131l l. also pictured in Fig. |9(a)| This scheme contains two instances 
of joint binning, therefore the graph representing the achievable region is a UDG and therefore we can define a CGRAS for 
this transmission scheme. 

We can derive the encoding errors only at decoder 1, since the errors at decoder 2 can be obtained by symmetry. 
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TABLE XIX 

Decoding error of event table for the CGRAS in (106), also pictured in Fig. |8(a)| 







Ui-,1 


C^2->{1,2} 




Dl 


m 








D2 


m 




/ 


Pyi\U2^[1,2} 


D3 


/ 


/ 




■Pyi|(7i^i,[/i_{i,2} 



Using the representation of the encoding error events in Tab. IVI-C2I both using Th. IV.4I and Th. IV.61 we obtain: 



+ i?i^{i.2} > nUi^i; (7i_{i,2}), (140) 

and by symmetry we also obtain 

i?2^2 + i?2^{l,2} > I{U2^2] U2^{1,2})- (141) 

We can use Tab. IXIXI to derive the bounds on the codebook rates using sequential decoding: 

il^{l,2} +^2^(1,2} < ^(n;J/l^{l,2},J/2^{l,2}IQ) (142a) 

ii^{i,2} < I{Yi; (7i^{i,2}|(72^{i,2}|Q) (142b) 

Li^i + £i^{i,2} < I{Yi; Ui^i, C/i^{i,2}|(:^2^{i,2}) + HU.^i; C/i^{i,2}, Q) (142c) 

i2->{i,2} < /m;(:^i^i,c/i^{i,2}|(:^2^{i,2},Q) (i42d) 

as well as using joint decoding 

Li^i + ii^{i,2} + L2^{U2} < I{Yi; Ui^i, Ui^[i^2},U2^{i,2}\Q) + HUi^i; f/i^{i,2}|Q) (143) 

Li^i + Li^{i,2} < I{Yi; C/i^i, C/i^{i,2}|t/2^{i,2}, Q) + HUi^i; C/i^{i,2}IQ) (144) 

^2^(1,2} < /(ri;C/i^i,C/i^{i,2}IQ) (145) 



Note that the rate bounds in ( |142d| l and (|145l l actually correspond to the error bound where the intended messages are 
correctly decoded, but the undesired message ^2^(1, 2} is wrong. This is not a decoding error, as pointed out in Rem. IV. 131 
and can thus be dropped. 

The FME of the binning rate bounds and the codebook rate bounds, without the constraints of (1145b . yields 



i?i^{i,2} + < /(i^i;(7i^i,(7i^{i,2}|t/2^{i,2},Q) (146a) 

i?2^{l,2} + i?2^2 < /(il^; (72^2, (72^(1,2} I t/l^{l,2},Q) (146b) 

i?l^{l,2}+i?l-l+i?2^{1.2} </(i^i;(^l^l,t^l^{l,2},t^2^{l,2}IQ) (146c) 

i?2-.{l,2} +i?2^2+i?l^{1.2} < I{Y2;U2^2,U2^{l,2},Ul~,{l,2}\Q) (146d) 



We can now perform the FME using the equality in ( |129t to obtain again the region in ( |139l l. 
3) Superposition coding and binning based scheme for the IFC: 

Let's consider first the scheme with only superposition in (1133b . also pictured in Fig. |9(c)| which. This scheme satisfies the 
conditions of Th. IIII.ll so we indeed have a CGRAS. 

This scheme is obtained by simply combining encoding and decoding error events of the two previous schemes to and thus, 
for this reason, we again obtain the achievable region in the Han and Kobayashi region in (1139b . 

VII. Conclusion 

In this paper we present a new general achievable rate region valid for a general class of multi-terminal networks. This 
achievable scheme employs rate-splitting, superposition coding, and binning, and generaUzes a number of inner bounds and 
techniques that have been proposed in the literature. This achievable scheme may be represented using a graph representation 
that allows for a quick comparison between transmission strategies and a simplified derivation of the resulting achievable 
rate region. This paper attempts to establish a general tool to derive achievable rate regions for multi-terminal networks 
which contain all standard random coding techniques. A subject of ongoing research is whether there exists a combination of 
encoding strategies that yields the largest achievable region among all possible transmission strategies (within the proposed 
framework). It is commonly believed that superposition coding enlarges the achievable rate region relative to codebooks 
derived from conditionally independent codewords. The same conjecture holds for binning but is less clear that this is the 
case when considering the union over all the possible distribution of the binning RVs. We believe that the general formulation 
for achievabiUty schemes and corresponding regions proposed here is a powerful tool to resolve these conjectures, with the 
ultimate goal of determining optimal coding strategies for general channels. 
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Appendix 

Since e and S from (O in Th. III. II can be driven to zero by considering the limit for N ^ oo and e — !■ 0, we often disregard 
this term from the expressions and simply use: 

P#(a;^) = (147a) 

|Tf|=2^^W (147b) 

P[X^ e Tf (Px)] = 1 (147c) 

for iV — > oo and e ^ 0. This is done for ease of notation and it is possible to show that the results derived in the paper stiU 
hold when using the inequalities in 



A. Proof of Thm.\nn\ 

Part I: Assumption |2] guarantees that the graph representation is a chain graph 

To show that the graph representation of a general achievable scheme can indeed be modeled using a chain graph Markov 
model, no directed cycle can exist in the graph representation. 

We prove this by contradiction: let's assume that a directed cycle of length n exists in the graph representation of a general 
achievable scheme and that Assumption |2] holds for the graph 

Uh^h < Ui^^h < ■■■ < C^i„^„ 

where < represents an edge obtained either by superposition coding or binning. Both superposition coding and binning require 

ii C so the directed cycle can exist only for i = ii = i2 = . . . = in- But by Assumption |2] the RV known at the same 

set of encoders i do not form directed cycles, which contradicts the assumption. 
Part II: Assumptions [T] and |3] guarantee that the chain graph is Markov equivalent to some ADG 
To show that under Assumptions [1] and |3] the chain graph corresponding to the representation of a general achievable scheme 

is Markov equivalent to an ADG we utihze the following theorem. 

Theorem A.l. [27,, Prop. 4.2] Let G = {V, E) be a chain graph, then the following are Markov equivalent 

• G is Markov equivalent to some (not necessarily unique) ADG 

• (G'cI(t))™ is decomposable for all the chain components r £ T(G') 

We refer the interested reader to ll27l for a more general study of equivalence between ADG, UDG, and chain graphs as 
well as a more complete definition of the basic concepts in the following proof. 

In a chain graph 9, the chain components r e T are the set of all the connected components obtained from S when removing 
all the directed edges. c1(t) denotes the closure of the chain component t G ^^iG), which is obtained by adding the boundaries 
of the chain component, i.e. c1(t) = bd(T) U t. (G'ci(r))™ denotes the moral graph of c1(t). This is the graph obtained by 
making the boundary of the set c1(t) complete, that is, by connecting all the nodes that have a common child. 

(G'ci(r))™ is decomposable if and only if it is chordal, that is if every cycle of length larger than 3 has a chord, an edge 
connecting two non-consecutive elements in the chain. 

Since the connected components are complete and the nodes in connected components have the same set parents, (G'cI(t))"' 
results in a complete graph, which is also chordal. 



B. Proof of Th.\nn\ 

The theorem can be proved in a similar manner to Th. 1111.11 The proof of Th. 1111.11 relies on the fact that the connected 
components are complete to apply Th IA.ll Since the connected components of the graph S{V,SUB) are complete, so are the 
connected components of the subgraph Si{V^,S^ UB^) and thus we can apply Th IA.ll to the subgraph Gcal{V^ UB^) as 
well. 



C. Proof of Th.\!vJ] 

To prove this result we extend the original proof of Willems for the MAC channel ll40l to the general setup of the CGRAS. 
Consider a general random encoding function 

^ Px^liUi^j s.t. k G Si},Q- (148) 

We next show that the deterministic function 

Xk =/x«({C/i^j s.t. fc G 5i},g,A^'=) (149) 
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for 

^^^-u(o,i), i.i.d.y k e {i...Ntx} 

has the same deterministic structure as in ( 11481) . 

TV 

Without loss of generality let both Xj^ and X/^ be defined over a subset of the integers 



= {i,2...\Xkr} 



7Vi 



and let i?^*" be defined as 



— N 

Encoder k sends X = I if 



Bn''{I) - J2j<i Px^\{u^,J s.t. fces,},Q(-^) I ^"^^ 



N 



B^' {I - I) < A""" KB^'il). 



By construction this event has probability 

7X, 



B^-{I) - B^^il - 1) = Px-i{u.„ s.t. fcesa^gU) 
while clearly being a deterministic function of the form 



-N 



Suppose that the random encoder in ( 1148b can achieve a target probability of error P^, that is: 



Pe ~ max 

(i.j)e{i...2"Tx}x{i...WR,},2eSj 



Wl. ^ W,,, > / P(A = [ai, 02 . . . aTx])P - ^ W^,,, |A 



Vz e 5, (150) 



where A, the set of A^*" , for each transmitter k 



A = {A-^^ fc G {l-.iVTx}}. 

The fact that the probability of error can be attained means that there exists a specific choice A = [ai, 02 . . . otx] for 
which 



By fixing A = A in the random encoder in ( 1149b we obtain a deterministic encoder that has a smaller error probability than 
the random encoder in (1150b . 



D. Proof of Th.E£ 

For a set S C Sb for which (|45] l holds, the probability of an encoding error P[Es] can be expressed as 



bi^j, (ij) e S, s.t. L/i^j e (p^ 



cncodingi S 



(151) 



where f'cncoding|s defined in ( 149 a| ) and is the distribution of the CGRAS given that the set {C/i-»j, (i,j) £ S} has been 
correctly encoded. Considering that the encoding of the codewords in S has failed though, the actual distribution of the 
codewords is ^'codcbook|s defined in (I49bb . 

An error in encoding the codeword U^^^ E S implies an error in encoding U^^^^ for all such that ?7i^m~C^i^j given 

that no choice of b\^in can impose the desired typicality condition. With this consideration we write: 



bi^j, (i, j) e S - rootB, s.t. C/i^j iwi^j,h^i) e Tf (P^ 



cncodingi S 



where S — roote is defined in (l481 t. We proceed now with bounding the probabiHty of error: 



n {Ui% (iJ) e S-rOOtB} ^Tf (PencodingIS) 

(i,j) e S — roots 

2"(E(i.j)es-„otB«i^j) 



[Es] < I 

(iJ 

< (1 - P [{UH^, (i, j) e S - rootB} e Tf (PcncodingIS)]) 



(152) 

(153a) 
(153b) 
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where ( |153b| i follows from the fact that the codewords are obtained with i.i.d. draws. Using the covering lemma of Th. IV. 11 
we bound the probabiUty term P lu^'l-., (i, j) £ S - rootBj G (Poncoding|S) as: 



[{Ui%, (ij) G - rOOtB} e Tf (FencodingIS)] < ^ 

^ 2^E[l0g P„„,odi„g|s] 2-JVE[l0g Pcodoboo 



[{uHj, (iJ) e S-rootB}] (154a) 



-ATI 



C.L. 



(154b) 
(154c) 

where the expectation is taken over the elements in S — root. The term I^ '^' as defined in (l47T l measures the statistical distance 
between the codebook distribution and the encoding distribution for the codewords that have been incorrectly encoded. 

We can apply the covering lemma to (|154t because the codewords in S — root are conditionally independent given the 
codewords in S and thus the encoding error for each bin is independent from the encoding error of all the other bins. If this 
were not the case the error events would be correlated and it would not be possible to apply the covering lemma and instead 
we would have to employ the mutual covering lemma. 



E. Proof of Th.^ 

We start by evaluating the term E [K] : 

E[K]= J2 IP[^b,.j = l] 
(iJ)eSB 



We now bound the term 



encoding. 



(155a) 

(155b) 
(155c) 



P [{C/i^j(«;i^j, 6i^j G bi^j} G Tf (Pencoding)] - E ^ t^i-j' j) ^ 

(Pencoding) 
= 2-^Icod°bo"„l 



(156a) 
(156b) 



Note that the term I™dcboo1c depends only on the codewords with a bin index (i, j) G Sb, since only through binning 

we can change the distribution of the codewords between the codebook generation and the encoding procedure. 

The evaluation of the term Var[K] is more involved than the calculation of E[K] and it requires to consider the correlation 
between two events 

for any possible (bj^j, bi^j). This term does not depend on the specific sequences bi^j and bj^j, but only on whether a pair 
of indexes in the two sequences are equal, i.e. foj^j — bi^j. This step is somewhat similar to the proof of Th. I V.4I and the way 
in which Es is defined: despite this we are not able to draw a strong parallel between the two derivations. 
To evaluate Var[K] we write: 



Var[K] = E[KK] 



E 



(157a) 
(157b) 

(157c) 



We can simplify the RHS of (I157cl i by noticing that a number of terms of the summation are equal to zero, that is 

P[/Vb.^, = 1, i^b.^, = 1] = IP[-^b,^, = l]IP[A'b.^, =1] ^ (158a) 

{{U,%{w,^iM^i), G bi^j} ^ Tf } ± {{[/if 6i^), e bi^j} ^ jf } (158b) 



The condition in (I158I I is verified in two cases: 
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• all the indexes and are different, 

• some of the indexes bi^j and bi^j are the same but they belong to different codebooks, that is when b\^m ^ 6i^m for 
some ?7i^j~?7i^m- 

This follows from the fact that the codewords are independently generated and indexed according to ( [37l ) and thus any 
difference in the indexes of dJTj i yields independent codewords. The condition for which the ( 11581 ) occurs can be formally 
expressed as 



3 6i^j,6i^j, s.t. = 6i^j =^ b\^^ = bi^m V (1, m) s.t. , Ui^m<Ui- 



(159) 



In the following we restrict our attention to the sequences bi^j such that this condition ( 11591 ) holds. We now continue to bound 
the error probabihty as; 



Var[K] = 5] J2 

bi-j bi^j, (159) 

= E E 

bi^j, (159) 

P[^b>..=l]P[^b>., = l|^b,.. = l 

= E E 

bn b.^j, (159) 

E[^b,.JP[^5,^^=l|i^b,.,=l 

=^-E E 

bi^j hi^j, (159} 



^b. 



l|^b,.,=l 



(160a) 

(160b) 
(160c) 

(160d) 
(160e) 

(160f) 
(160g) 

(160h) 
(160i) 

(160j) 



Consider the term 



^b,., - ll^b,., = 1 



in the RHS of ( 160j i: since the codewords are generated in an i.i.d. fashion, 

this term does not depend on the specific values of bi^j and bi^j but solely on whether fei^j = bi^y 

With this consideration we can rewrite the summation not in terms of the sequence of bin indexes bj^j and bi^j as a sum 

over all the patterns in which the indexes in the two sequences can match each other 

We now link the notation of this theorem with the notation of Th. IV. 11 by noticing that condition ( |159l l can be rewritten in 

an equivalent form as ( |45] ) by defining S as: 



S = {(iJ), 



(161) 



in fact one readily notices that ( |159t does not depend on the specific sequence bi^j but only on whether — bi^j and thus 
( 11591 ) for the set bj^j translates to condition ( l45T l for the set S. 
With the convention in (|161| l we can write 



l|^b,.,-l 



Var[K] < E[K] • ^ E ^ 

bi^j bi^j, (159} 

< EficJ- ^ 2^('.J)es^'-jp [Es\Kb,^^ = l] , 



(162a) 
(162b) 



bi_j ScSb, ED 

where (I162bb follows from the fact that each pattern S appears at most 2^(' j)«^s ^'"j, and where P [£'s|i^bi^j = l] is defined 
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as: 



[Es\Ki,,^, = 1] = P [{U(l^{wi^i,bi^i), £ bi^j} £ Tf (Pencoding) | { t/j^j (u^i.j , ) , £ bi^j } £ Tf (Pencoding) , 

(163a) 



- V (i,j) £ S 



(163b) 



{C/i^jK^j,6i^j),6i^j £ bi^j, (i,j) £ S} U £ bi^j, (ij) £ S} £ Tf (Pencoding) 

(163c) 



I {Utii{wi^i,bi^i), £ bi^j} £ Tf (F, 



encoding 



)] 



{C//!j(u;i^j,&i^j),&i^j £ bi^j, (i,j) £ S} £ Tf [P^ 



cncodingi S 



(163d) 
(163e) 



Since joint typicality implies marginal typicality, the codewords in S have the desired typicality properties for encoding while 
the codewords in S can be correctly encoded only if the binning rates are sufficiently large. The distribution that we must be 
able to impose on the codewords in S is again the distribution Poncoding|s <l49al ). In fact the codewords in S appear to be 
generated according to the distribution J2u- ■ (i j)s ^'^'^ ^^^^ '■^^ remaining codewords must appear to be generated according 
to the distribution Pcncoding|s ' which is the distribution Pcncoding condition on {C/i^j, (i, j) £ S}. The term P [i^sl-K'bi^j = l] 
can be bounded in a similar way as ( |154t by using the mutual covering lemma of Th. IV.2I 

P [Es\K\,^^ = 1] = P [{C/f j, (iJ) £ S} £ Tf (P_,,„^ 

P[{C/i.j£S}] 

= 2^^N ^o„cod.„,|s] 2-^E[iog (164^) 

.rjM.C.L. 

= 2~^'s , (164d) 



(164a) 
(164b) 



where the expectation is taken over the elements in S and where Po„coding|s -fcodobook|s defined in ( 149 a| ). Note that the 
term c jjj ( |5gb| | and Ig in (|47] | is in the set where the expectation is considered: c l. consider all the elements 
in S while in I^'^' we only consider the elements in S — root. 

We notice here that since the elements in S are not conditionally independent given S, it is not possible to use the covering 
lemma and it is instead necessary to use the mutual covering lemma, which allows for the codewords with a bin index to be 
dependent. This was not the case in ( 11541 ) for the proof of Th. I V.4I where the covering lemma is sufficient. 

We can now return to (|59c| i and use both ( 1155b and ( 1164b to bound the probability of encoding error: 



'[K = 0] < 



E 



) 



Therefore the RHS of ( 11651 ) goes to zero when ( |57] l holds 



E[K] 

bi^j SCSb, ED 

bi^j SCSb, |45} 



) e s 



^M.C.L. 



(165a) 
(165b) 
(165c) 



F. Proof of Th. \V8\ 

For the set S C Sd such that ( |66] l holds, the probability of decoding error P[P'|] can be expressed as 

p[i?|] = p [a (wi^,bi^j) ^ (1,5) s.t. (y^{L^i^j(^5i^j,6i^j), (i,j) £ s^}) £ Tf (ij) £ s^})' 



(166) 

An error in decoding the codeword £ S implies an error in decoding for all such that C/i^m~C^i-!-j or 



Ui^m < Ui^j given that Ui^m depends on Wi^j and fei^j. With this consideration we write: 

P [^s] < P [{Y,'', {f/i^jCw'i^j, e - rootD}) £ Tf ({F., C/i^j, V (i, j) £ - rootD})] , 



(167) 
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that is, the probability of the decoding error event Ds^ is upper bounded by the probability that the elements in S — root^, 
defined as in ( |63] l. are incorrectly decoded. Continuing the above series of inequaUties we write 



U (yf , e - rootD}) e Tf (P^^^ 



cncodingi S 



(168a) 
(168b) 



where ( |168bt follows from the fact that codewords are obtained from i.i.d. draws. Using the packing lemma of Th. IV. 3 1 we 
can bound the probability of the term P [(^z^, £ — rootD}) e (^n, encoding)] as: 



, {f/i.j(«;i^j, G - rootD}) e Tf (P^^ 
< E ^ [{y., LZ/Ij, (i, j) e - rootD}] 



"^"{Py^. cncodingis) 
2AfEy^_S-- -rootD [log P^,^^ 



where the expectation is taken over the elements in — rootD and where 

□ -FkKC/i^j V (i,j)}-Pencoding 



Yz ^ decoding|S 



S(i,j)GS -foncoding 



(169a) 
(169b) 

(169c) 
(169d) 

(170) 



The term Ig ^'^ as defined in ( IV. 8 1 measures the statistical distance between the distribution of the channel output and the 
codewords in — rootD in the case of correct and incorrect decoding given the correctly decoded codewords in S. 



G. Proof of Th.^TJl 

Consider again the factorization of the decoding error probability in Th. IV.8I another approach in bounding the probability 
of decoding error in ( 1167b is: 



U (y,^, {c/i^j(u;i.j, e s}) e Tf (p^^,,_ding|s) 



yf , {C/i^j(u;i^j, 6i^j) e S^}) , e Tf [Py^ 



3ding|S 



(171a) 



(171b) 



where (I171bb follows from the fact that codewords are obtained from i.i.d. draws. Using the packing lemma of Th. IV. 3 1 we 
can bound the probability of the term 



(y/,{[/i^j(«;i^,6i^j) e S-}) G Tf P,. 



V~ .cncodingi 



^) 



as: 



(y,^,{f/i^K.j,6i.j)eS^})eTf (p 



Yz , encoding! S 



p 



■^"(-Pyj.encodingis) 
^ 2^Ey,,s[logP^^_ c„codi„g|sl2-^^^-s[log[P^^_ dccod.„g|S 



S^j^cncoding ({C^i^j G ^1) -feeding ({t^i^j G ^^1) 



(172a) 
(172b) 

(172c) 
(172d) 



where the expectation is taken over the elements in S^. The term Ig ^'^ as defined in (l83b measures the statistical distance 
between the distribution of the channel output and the codewords in S in the case of correct and incorrect decoding given the 
correctly decoded codewords in S. 



